Embed Size (px)
Citation preview
CALIFORNIA STATE UNIVERSITY
MONTEREY BAY
Zoonotic Disease Training
CAPSTONE Report
Submitted in partial satisfaction of requirements of the degree of
MASTER OF SCIENCE in
Instructional Science and Technology
Brianne Fitzgerald
November 18, 2017
Capstone Approvals: (At least one advisor and capstone instructor should approve)
_________________________ ___________________________ _____________
Advisor Name Signature Date
_________________________ ___________________________ _____________
Capstone Instructor Name Signature Date
2
Table of Contents
Executive Summary ........................................................................................................................ 4
Introduction and Background ......................................................................................................... 5
Needs Analysis ....................................................................................................................................... 5
Learner Analysis ..................................................................................................................................... 6
Environmental Scan ................................................................................................................................ 6
Solution Description ....................................................................................................................... 8
Project Goals ........................................................................................................................................... 8
Learning Objectives ................................................................................................................................ 8
Training Solution .................................................................................................................................... 9
Learning Theories ................................................................................................................................. 10
Instructional Principles ......................................................................................................................... 12
Media Components ............................................................................................................................... 13
Challenges ............................................................................................................................................ 14
Methods and Procedures ............................................................................................................... 15
Design and Development ...................................................................................................................... 15
Implementation Plan ............................................................................................................................. 17
Resources .............................................................................................................................................. 17
Timeline ........................................................................................................................................ 18
Evaluation ..................................................................................................................................... 19
Methods ................................................................................................................................................ 19
Learners ................................................................................................................................................ 20
Expected Outcomes .............................................................................................................................. 20
Results ........................................................................................................................................... 20
Usability Results ................................................................................................................................... 20
Learner Experience Results .................................................................................................................. 21
Lesson Objectives Results .................................................................................................................... 22
Revisions .............................................................................................................................................. 23
Data Analysis ........................................................................................................................................ 23
Data Summary ...................................................................................................................................... 25
Conclusion .................................................................................................................................... 25
References ..................................................................................................................................... 27
3
Appendix A ................................................................................................................................... 28
Appendix B ................................................................................................................................... 29
Appendix C ................................................................................................................................... 30
4
Executive Summary
As an accredited member of the Association of Zoos and Aquariums (AZA), the
Monterey Bay Aquarium (MBA) must conduct zoonotic disease training for employees and
volunteers of the Husbandry Division (animal care team). A zoonotic disease is a disease spread
between animals and humans under normal circumstances. They can be transmitted from frozen
food items offered to the aquarium’s animals, or transmitted directly from the animals
themselves, or through their aquatic environments.
The Husbandry Division is the department that curates, maintains, and manages the live
animal collection and has a total of 175 employees and volunteers collectively. This combined
group, referred to as workers for the rest of the document, requires training on zoonotic diseases
every two years. The staff veterinarian is the only subject matter expert (SME) and sole provider
of the current in-person training sessions. He faces the scheduling challenges of trying to deliver
the training course in a consistent manner to this large audience. The veterinarian and the
management team requested a comprehensive, modular, online training course that would fit
their division’s needs in an effort to increase zoonotic disease awareness and introduce a new
safety requirement (wearing gloves when handling food items). The veterinarian requested a
course design that could be customized for other departments within MBA and for other AZA
organizations.
After reviewing the needs analysis results and conducting the environmental scan, I
determined that a blended training intervention would be the best solution. The blended program
includes an instructor-led presentation followed by an online course developed in Adobe
Captivate. The instructor-led presentation introduces a new mandatory standard of practice for
food handling safety and will address the employees’ concerns regarding the adoption of the new
requirement. After participating in the instructor-led session, workers would then take the online
course which subsequently serves as the organization’s compliance training for years to come.
This report includes an overview of the proposed blended course and then details the
Adobe Captivate online course development and assessment process. The online course is the
deliverable of this Capstone Project. Formative and summative evaluations were conducted for
the asynchronous Zoonotic Disease course. The SME conducted a review and gave approval of
the content outline and storyboard. A representative user group tested the course for functionality
and participated in the summative assessment.
5
Introduction and Background
Needs Analysis
Personal interviews were conducted with Husbandry Department employees and
managers, the veterinarian, and two members of the Safety Department. Documents reviewed
included job descriptions, AZA training standards, worker onboarding checklists, and injury
reports.
According to M. Murray, DVM, “this department [Husbandry] includes members with
the highest level of animal exposure at the aquarium, and therefore, a high zoonotic disease
exposure risk. In addition to exhibit animals (which include mammals, aquatic species, reptiles,
and amphibians), members of the team are also exposed to wild ‘nuisance’ animals that arrive on
site, such as raccoons and pigeons” (M. Murray, DVM, personal communication, September 10,
2016).
The main performance issue discovered was the employees’ lack of adopting expected
safety measures. A gap existed between what the employees “should” know from training and
the behaviors they were demonstrating. Department managers and the veterinarian wanted
workers to wear gloves in the food preparation room and anytime they were handling food items,
however, that expectation had not been communicated as a mandatory performance standard.
The veterinarian viewed this issue as a managerial responsibility; he viewed his only role as
providing disease exposure information and suggestions for best practices. “The training goal is
to provide a greater understanding of the risks inherent in working with wildlife and to provide
the mechanisms to mitigate risk (Dr. M. Murray, DVM, personal communication, October 16,
2016). Managers expected employees to do the “right thing”. This was an ill-defined expectation
which was supposedly achieved by workers using their ‘common sense’ and disease awareness
knowledge they may have gained from the existing zoonotic training course. This training
solution was designed to increase the workers’ awareness of zoonotic diseases and to reinforce
clear performance and reporting expectations. The managers were consulted during the
development and implementation phases as their involvement was instrumental to successful
training outcomes.
The lack of employees’ consistently adopting the desired safety practices was attributed
to both internal and external factors. Employees viewed the wearing of safety gloves as an
6
impediment to conducting their daily activities in an efficient manner. Some employees also felt
that they would earn the stigma of being a ‘wimp’ amongst their peers for choosing to wear
protective gloves. There were still others who did not consider wearing gloves because they had
not been injured in the past. The development and addition of managerial support systems would
engender a positive safety culture and help achieve the prescribed safety behaviors.
Learner Analysis
The Husbandry Division is a homogenous team with similar backgrounds in biology and
marine science whose ages range from 25-40 years of age. The husbandry volunteers are a more
diverse group in relation to their animal care experience and science backgrounds. Their ages
range from 20-65 and they are also more ethnically diverse. Both employees and volunteers
represent a wide range of longevity at the aquarium and therefore different levels of technical
skills and historic, departmental knowledge.
Workers with more than five years at the aquarium have a higher likelihood of being
exposed to zoonotic diseases training, however, the delivery has been sporadic and
undocumented. Prior training is not necessary for the zoonotic training course. Employees have
backgrounds in the biological sciences with a minimum of bachelor’s degrees. Volunteers vary
in their degree of scientific knowledge and animal-related work experience.
Workers have varying degrees of comfort with technology which does not necessarily
impact their success with the online course but may necessitate higher levels of support. The
organization has shifted to more online training with the rollout of a new learning management
system (LMS) and both employee and volunteer groups have slowly been exposed to eLearning
as a training method.
Employees are onsite in the main aquarium building and each has their own workstation
and computer. Volunteers of each workgroup need to access the training from their personal
computers, tablets or smartphones.
Environmental Scan
Dr. Murray requested information from other AZA institutions. They provided their
current zoonotic diseases training information, methods of delivery, and training documents A
review of this information and interviews with their training providers revealed that training
varies both in how it is designed and/or how it is delivered within different organizations.
7
However, they all followed general outlines which include a general overview of zoonotic
diseases, examples of common diseases, signs and symptoms, prevention, and what to do when
injured. The Long Beach Aquarium had the most robust training plan including the establishment
of a Zoonotic Diseases Committee which communicates with local hospitals and care providers
to update them on what to do if a person reports to them with symptoms. Chicago’s Shedd
Aquarium converted their training into an online interactive course using Adobe Captivate. They
have created a library of instructional tools for awareness and prevention which are assigned to
new hires who fall into different categories of exposure risk. Their veterinarian then delivers
more advanced annual presentations to their different work groups. Communications from all
training providers at these institutions indicated that they were interested in a course designed for
use by all AZA institutions.
A review of the AZA accreditation standards did not produce any requirements or
recommendations for training content. The standards simply stated that zoonotic disease training
and preventative procedures must be in place. “Institutions must train appropriate staff in
methods to prevent zoonotic disease” (AZA standards, 2016). The National Association of State
Public Health Veterinarians (NASPHV) has prepared a Compendium of Measures to Prevent
Diseases Associated with Animals in Public Settings, which should be followed by institutions
presenting animals for public contact. The NASPHV recommends that local and state public
health, agricultural, environmental, and wildlife agencies use these recommendations to establish
their own guidelines or regulations for reducing the risk for disease from human-animal contact
in public settings (Williams, Scheftel, Elchos, Hopkins, & Levine, 2013). The AZA standards
require training, the Public Health Veterinarians group provide materials and statistics that can
be included in training design, but individual organizations and agencies are responsible for
developing their own training.
The Journal of Safety Research, Vol. 3, 2002 included an organizational study that was
conducted to determine which management factors were most effective in reducing workplace
injuries. It empirically confirmed that safety training for employees at the start of hire could help
reduce injuries. “The most important finding of this study is that when organizations take
proactive measures to protect their employees, the company derives a financial benefit in
reduced lost time and workers compensation expenses” (Vredenburgh, 2002). This study also
demonstrated that providing training in itself was not adequate. Organizations must verify that
8
the safe practices taught in the classes are being implemented in the work areas (Vredenburgh,
2002). The study went on to discuss that key factors for reducing injuries include changes to the
organizational culture that would perpetuate messages of greater care for those who get injured,
instead of messages that convey the importance of saving money and reducing lost time from
injuries.
Solution Description
Project Goals
The goal of this project was to revise the current zoonotic disease training into an online
learning course based on instructional design principles and processes. The final project will be
utilized in the organization for compliance training in the Husbandry Division and will be
customized for delivery to other departments. It can also be modified for use by other AZA
organizations if it proves successful over time. The intended outcomes are for workers to choose
to wear gloves every time they handle animal food items, practice proper hygiene for disease
prevention, and to have administrative support of those outcomes through managerial systems of
accountability.
The training course accomplishes this by:
• providing definitions for common terms associated with disease transmission,
• heightening workers’ awareness of common diseases prevalent in the aquarium,
through an overview of common diseases and their hosts in the aquarium collection,
• introducing current zoonotic disease statistics,
• sharing real images of zoonotic injuries from the relevant work environments, and
• introducing new requirements from management with additional accountability
measures.
Learning Objectives
Terminal Objective: Workers will use proper protective equipment when handling animal food
items (affective domain).
Enabling Objectives: Online Course:
1. Given a list of possible answers, workers will be able to identify the correct definition
of zoonotic diseases (cognitive domain – recall).
9
2. Given a selection of disease scenarios, workers will be able to differentiate those
which are infectious and those which are communicable (cognitive domain-
evaluating).
3. Given a list of options, workers will be able to identify those that are disease
transmission routes (cognitive domain- recall).
4. Given zoonotic disease scenarios in the workplace, list the proper steps to follow that
would prevent disease transmission (cognitive domain- recall).
5. Given three scenarios, workers will be able to identify which elements of each
scenario pose a heightened level of risk of infection (cognitive domain- evaluate).
6. Given a list of options, select the proper order of steps for reporting suspected
zoonotic disease symptoms (cognitive domain- recall).
Enabling Objectives: Instructor-led Session
1. In a group setting, describe and discuss personal and departmental barriers to
adopting the glove-wearing mandate (affective domain).
2. Given common cultural barriers to compliance, propose solutions that could be
adopted by the department and colleagues to mitigate those barriers (cognitive
domain- evaluate, analyze, create).
3. State the most significant task that can be practiced at work to prevent zoonotic
disease transmission (cognitive domain- recall).
4. Given a pair of nitrile gloves, correctly perform the steps of removing them without
contamination (psychomotor domain).
Training Solution
The product created for this Capstone Project is the online zoonotic disease training
course. The instructor-led course outline and activity have been developed and will be further
refined post-graduation. The overall package will be consolidated and rolled out together,
beginning with the instructor-led portion followed by the online training course. The online
training course will then serve as the division’s bi-annual compliance training. The content
design presents an overview of zoonotic diseases, where they are encountered, and how to
prevent disease transmission. Relevance is established by portraying work-related images and
scenarios that can result in injuries and includes the veterinarian (a respected colleague) as the
10
spokesperson. An element of “shock factor” is involved by graphically depicting real zoonotic
disease symptoms and injuries as a measure to encourage employees’ use of gloves.
Online Zoonotic Diseases Course Outline
1. Course Introduction and terms
a. Dr. Mike introduces the course, outlines course objectives, and defines
zoonotic disease.
b. Current statistics on zoonotic disease transmission.
c. Topic relevance to the aquarium setting.
d. Terms to better comprehend how diseases are transmitted (infectious,
communicable, pathogen, agent).
2. Disease Transmission
a. Seven routes of transmission with explanations.
b. Situations in which individuals are at higher risk of exposure.
3. Description of Common Diseases found at the Aquarium
a. Type of pathogen.
b. Animal group/species where found.
c. Signs and symptoms.
d. Summary points.
4. Prevention
a. Review of heightened exposure risks.
b. Review of prevention measures.
c. Highlight new requirement compliance expectation.
d. Review proper reporting procedures.
e. Provide proper hand-washing method.
Learning Theories
Activity theory (developed by L. Vygotsky, A. Leont’ev, and S. Rubinstein in the 1930’s)
is a framework or descriptive tool for a system. “(It) considers the entire work/activity system
(including teams, organizations, etc.) beyond just one actor or user (Learning Theories blog,
n.d.). According to Laberge, MacEachen, and Calvet (2014) current occupational health and
safety (OHS) training approaches are based on a cognitive or a behavioral educational paradigm,
which is oriented to shaping the new worker’s attitude or behavior so that he or she will follow
11
OHS rules. This training approach generally involves an instructor-centered environment of
stating the rules and making attempts to develop safe attitudes.
The study by Laberge et al. set out to determine if those methods were effective with
young workers, or if a learner-centered approach based in activity theory would be more
effective. Through interviews of these “apprentices”, it was concluded that when learning new
skills at work the social environment can be both a learning resource and a source of constraints
(Laberge, 2014). Experienced co-workers can pass on useful information. Or, these co-workers
can give too much conflicting advice that becomes confusing and may go against normal
operating procedures. Designing training within an activity theory framework acknowledges that
activities (safe work practices) are accomplished within a social system and considers safety
culture and safety climate as affecting performance outcomes.
Another area of research on activity theory indicated that a more successful learner-
centered safety program would include worker’s knowledge and first-level supervisors in the fine
tuning of safety practices (Ripamonti & Scaratti, 2015). Safety teams generally approach
interventions based on statistical data of injury-related reports and will employ new, unilateral
safety practices as a response without including managers in the process. These safety practices
are often not conducive to the ever-changing nature of work activities, forcing workers to break
the rules in order to accomplish their tasks. Additionally, front-line managers will often accept
this type of conduct as they have a better understanding of the conflicting pressures employees
face when accomplishing their work. A broader perspective is needed to create more effective
safety compliance training. Organizational factors must be included with individual behavior
analysis.
The zoonotic diseases training design acknowledges the needed changes to the
organizational safety culture and worker attitudes toward safety by including managers and the
veterinarian in the analysis and design phases. The instructor-led portion will address the socio-
cultural barriers and will include facilitated discussions to encourage a positive safety climate.
Bloom’s Taxonomy of Learning Domains identifies three areas of learning categories:
cognitive, affective, and psychomotor. This training includes elements in the psychomotor
domain by showing workers how to remove gloves and wash hands properly. It includes
elements in the cognitive domain by having workers recall information and evaluate which
situations require PPE implementation. The affective domain addresses the emotional aspects of
12
learning including motivation, attitudes, and feelings. This training design attempts to motivate
employees to choose preventative safety measures. It contains elements which are based research
results known to influence the affective domain such as getting learners to be willing to listen to
the material, motivating them to participate with the material, connecting the material to their
value system, and helping them to resolve conflicts within their value systems. “If that can be
accomplished, then the knowledge will result in a new set of internalized values that controls
their behavior” (Clark, 2016).
Instructional Principles
The instructor-led session is critical to priming the audience for the online course. Adult
learning principles and learning motivation models will be utilized to prime the learners during
the classroom session. Negative attitudes toward imposing safety measures exist. Workers will
need to adopt new prevention practices. It is critical during the live session to acknowledge
workers’ attitudes, concerns and feelings regarding the new mandate and expose them to the
relevant impacts of not adopting the new requirement. Table 1 outlines the adult learning
principles and their applications followed by descriptions of the principles and their applications.
Table 1 Learning Principles and Instructional Strategies
Adult
learning
principle
Instructor-led session strategies Online Module strategies
Readiness • Engaging invitation to event
• Prior to session, send out assessment
survey of why or why not people
wear gloves
• Delivered by respected colleague
• Primed by information from live session
• Real case stories from colleagues including
images
• Current disease statistics
Experience • Sharing personal stories and
experiences
• Recognizing and sharing
concerns/resistance
• Common, scientific terms used in their field
and at the learner’s level
• Graphic depictions of real cases from
colleagues as well as other, similar
organizations
Autonomy • Group activity
• Group discussion
• Interactive design with learner control
Real case diagnosis
Action • Engaging with online module • Adopting new safety practice of wearing
gloves
• Accountability/Recognition system from
managers
Relevance • Introducing related work cases • Task-related scenarios and images
Table 1. Adult learning principles and instructional strategies designed to increase motivation and address
attitudinal barriers.
13
1. Readiness – Stolovitch and Keeps (2011) state that adults come to a learning situation with
their own priorities and attitudes and are ready to learn when they decide to open their minds
and spirits to it. Session invitations will prime participants for learning content and
meaningful benefits will be outlined which are relevant to the workers’ circumstances. This
will create an environment for learning in which the benefit for the workers is to avoid
serious health repercussions. Current statistics on zoonotic diseases from the Center for
Disease Control will also increase awareness and relevance.
2. Experience – Workers will have the opportunity to share their personal experiences with
zoonotic diseases at work or in the home. There will also be discussion and recognition of
their historic reasons for not wearing gloves when handling food items. By helping them to
recognize and share the barriers to the glove requirement, resistance can be diffused by
demonstrating empathy and understanding of past problems and work realities.
3. Autonomy- Workers will now be required to wear gloves when handling food items. This
does not leave much room for autonomy, or the choice not to wear gloves. They may still
choose not to, but there will be a higher level of scrutiny and accountability post training.
Autonomy will be established within the live session through group participation in activities
and through participation in group discussion. It is a self-paced course with real-world
elements including “drag and drop” elements of disease exposure risks and relevant images.
4. Action – Adult learners must have a means to directly and immediately apply what they have
learned in training to their work setting. It is incumbent upon the management team to
reinforce learning through direct observation, recognition, and consequences for adopting (or
not) the new glove-wearing protocol.
5. Relevance –One of the best ways for adults to learn is when the learning objectives are
directly tied to their own tasks. This is accomplished through scenario-based instructional
strategies. Knowledge checks are included using pedagogical agents posing work-related
problems for the learners to solve. Images from the aquarium are also included to further
establish relevance to on-the-job tasks.
Media Components
The online course was developed using Adobe Captivate. The veterinarian is the primary
spokesperson, introduces the course, and is the actor in the course introduction and in thr
14
instructional video on proper hand-washing methods. He is also introduced throughout the
course as an illustrated cartoon for personalization and consistency. Additional media includes:
• images and illustrations of disease symptoms, pathogens, and carriers,
• audio narration (myself, as a trusted colleague),
• characters from eLearning Brothers in the “Knowledge Checks”,
• closed captioning on all videos and the overall course, and
• an aquarium safety protocol included as a PDF document.
Challenges
The primary challenges encountered with this project included:
1. SME access. The veterinarian has a full clinical and traveling schedule. I was able to
get the major content review and design approval completed. I will need to conduct
another final review session with him post-graduation and prior to the training roll-
out.
2. Attitudinal barriers. The performance expectation of wearing gloves will require
behavioral changes for some. Currently, the veterinarian has stipulated the exception
to this rule when feeding sea otters. Some managers have reservations about this
exception to a safety requirement. The management team is currently (as of 11/7/17)
discussing this with the veterinarian to confirm the messaging. I cannot deliver the
course to the workers until I receive the managers’ decision on the requirement.
3. Organizational change. There has been a recent large-scale change in the upper
management of the department. This has left vacancies to fill, thus slowing the
process of discussions regarding the new glove-wearing requirement and ultimately
affects the implementation timeline.
4. Adobe Captivate. I am still learning what it can do. As I became more skilled in its
use, I was tempted to go back and redesign some design elements and activities to
make them more interactive and engaging. I had to settle for its current iteration to
meet school timelines for graduation. I have ideas to improve its design, apply the
aquarium’s brand elements and add more scenario-based instruction with branching.
15
5. Narration. Developing a method for adding narration to the course was challenging
because of the available resources. Sound quality was an issue. I eventually utilized a
process of audio capture with my Galaxy S7 audio recorder.
Methods and Procedures
I utilized the Human Performance Technology (HPT) Model as the method for
determining the blended learning intervention. I compared the desired worker performance with
what was actually occurring and identified the performance gap (some individuals not wearing
gloves). By conducting interviews, I determined that the workplace environmental factors and
the workers’ motivations contributed to the performance gap. The holistic approach of the HPT
model to workplace and performance analysis provided the vehicle for uncovering aspects of the
performance gap that I would not have discovered otherwise. I utilized analysis templates from
Rapid Instructional Design by George M. Piskurich as the basis for the interview questions.
Design and Development
The design of the online course follows the typical outline of the current zoonotic disease
training and those shared by the other aquariums. I reviewed content topics and the general
outline with the SME and confirmed the learning objectives for this particular audience. I used
the aquarium’s style guide as the basis for the overall design of the fonts, colors, and button
states.
The course has four modules- Definitions, Routes of Transmission, Diseases and
Symptoms, and Prevention. Module 1 is a course introduction and provides zoonotic disease
terminology and definitions. I developed the introduction video featuring Dr. Murray and a drag-
and-drop knowledge check to reinforce the information in an engaging way. Module 2 introduces
the routes of disease transmission. Definitions are revealed through learner interactions. Module
3 was developed first as it was heavy with media, content, and interactions. I accessed the
aquarium’s image database and conducted online searches for relevant and appropriate images.
The content for Module 3 was provided by Dr. Murray. I also gathered more detailed disease
information from the CDC’s website. Module 4 is a summary module which includes the proper
reporting procedures, reinforces the preventative measures, and reviews the new performance
requirement.
16
The following list is the general outline of the steps completed for this Capstone Project
and its major deliverables:
1. Course content review (SME)
2. Storyboard approval (SME/Advisor)
3. Course completion review (Advisor)
4. LMS upload and trials
5. Course instructions/assessment developed using LMS authoring tool
6. Functionality Test (Advisor/User group)
7. Summative Evaluation/Assessment (User group + 4 more)
8. Course Summary Report
Major deliverables
1. Introduction and definition of zoonotic diseases
• SME introductory video
• Terms defined
• Interactive knowledge checks
2. Routes of Transmission
• Images and audio narrative
• Images of heightened exposure risks
• Interactive knowledge checks
3. Types of diseases, signs, and symptoms
• Images of symptoms, audio narration
• Quiz questions
4. Prevention
• Review of prevention methods
• Instructional video of proper hand-washing technique by Dr. Mike
5. Assessment
• Pre-Post course survey and observations for formative evaluation.
• Course pre- and post- knowledge assessment
• Statistical analysis as summative report
17
Implementation Plan
I am the LMS administrator for the division and will deploy the course via emails which
will include access to the course in MBALearns. The course will be delivered after the facilitated
instructor-led session is conducted. During the live session, the SME, the Division VP, and the
Safety Manager will outline the goals of the new training course, align the course content with
departmental safety goals, introduce the new performance requirement, and provide an overview
of the training support and accountability system. I will guide the employee activity and facilitate
the discussions that will address employees’ concerns over the new standards.
The workers will then access the online course with their LMS account and complete it
within a two-week timeframe. Course compliance will be set for two years and will
automatically reset two weeks prior to the end of the compliance date. I will collect user data and
assessment information upon course completion to utilize for any revisions needed.
Resources
Costs associated with the course were minimal. Developer costs were associated with the
hourly rate of the employee designer (myself) for any design and development time spent during
work hours. Resources for the project are listed below.
• Adobe Captivate 9 for online course development (purchased)
• Camtasia for video editing (purchased)
• Litmos – MBA learning management system
• Onsite conference room for training session
• Microsoft PowerPoint for storyboard
• Microsoft Word for draft development documents
• MBA intranet/Microsoft Outlook for implementation communications
• Google Drive for group and SME document edits
• On-site access to workers for interviews/video shoots
• Video camera (MBA resource)
18
Timeline
I convened an initial meeting with stakeholders to establish
• consensus on performance standards to include in the course,
• managerial support and accountability systems,
• roles and expectations of stakeholders during the implementation phase, and
• to generate input and consensus on general content inclusion.
Table 2. Project timeline with completion dates
Table 2 Project Timeline
Item Description Status Timeline 2017
Course Content
Outline
Completed Completed SME review 8/22
Storyboard
development
Course outline with media
placeholders, navigation included
Completed in
PPT
SME review 9/15
Resource
mining
Images, videos, narratives Complete 9/16 - 9/20
Project Development
Phase 1 Master slide and template design
creation with navigation and quizzes
in Captivate
Complete 9/20 - 10/1
SME/Advisor review 10/3
Phase 2 Addition of media, final production Complete 10/2 - 10/20
SME/Advisor review
10/24
Phase 3 Publish and upload into LMS Complete-
closed
captioning issue
11/7
Testing
Development Pre- and post-course assessment and
surveys developed
Completed 11/1 - 11/5
Functionality
Tests
Users added to LMS/course
delivered to small user group (4)
Completed 11/7 - 11/14
Summative
Evaluation
Pre-and post-course assessments and
surveys conducted
11/7 - 11/24
Final Report Training design report with
evaluation results
Draft begun
Completed
11/9
11/20
19
Evaluation
The formative evaluation was conducted with a user group of both experienced animal
care employees and novices not employed by the aquarium. I did not involve many employees (3
out of 8) in the evaluations as I was faced with a barrier- the management team and the
veterinarian had not yet agreed on the exact nature of the performance requirement of wearing
gloves. As of this report, a debate exists whether to require employees to wear gloves when
feeding sea otters. I did not want to add to any confusion by testing the course with employees
knowing that I may need to change the terminal learning objective.
Methods
The assessment instruments and the training module were accessed during the same
observation sessions. The learners utilized their personal computers, some with Macs and others
with PCs. The internet browser was either Chrome, Firefox, or Safari (MBALearns in not
compatible with Internet Explorer). Learners were emailed links to the course surveys and
assessments, and the link to MBALearns. They completed the pre-course survey and pre-course
assessment prior to the observation periods. I then met with everyone individually to observe and
facilitate the training course. I took observation notes and input during those sessions and then
asked them to complete the post-course surveys and assessments on their own.
The surveys were created in Google Forms and designed to gather learners’
demographics, feedback on the usability of the training course, and feedback from their
experiences with the course. The post-course survey listed questions in three categories:
usability, course feedback, and lesson feedback.
The assessment questions were also created in Google Forms (Appendix B). The pre-
course assessment was designed to gather prior knowledge on the topics covered in the course.
The same questions were listed on the post-course assessment to serve as a measure of learning
which may have taken place as a result of lesson participation. There was an additional quiz built
into the MBALearns course that was not included in the course evaluation design. Each
participant did take the quiz so that I could collect their feedback on improving the questions for
clarity and course consistency.
20
Learners
The participants chosen for this evaluation represent the target audience of predominantly
female workers with an age range of 25-60 years. The target training audience has a diverse
background of zoonotic disease and biology knowledge as this course is designed for both
experienced animal care employees and for novice volunteers.
Demographic results from the pre-course survey indicated that
• the learners for this evaluation were predominantly female (7 out of 8),
• the average audience age was 46.2,
• 3 out of 8 learners were animal care employees,
• 50% of the learners had been or are currently volunteers at the aquarium, and
• 3 out of 8 had taken the aquarium’s current zoonotic disease training.
The learner demographics accurately matched that of the target audience as there are more
volunteers than employees in the Husbandry Division.
Expected Outcomes
Upon completion of the training module, it was expected that data results would show an
increase in knowledge of zoonotic disease exposure and of prevention methods after
participating in the course. The learners would also recognize the new performance expectation
of wearing gloves whenever handling food items.
Results
Usability Results
Four usability questions were asked in the post-course survey. The responses were
quantified by adding a 1 for “Strongly Disagree” and a 5 for “Strongly Agree”. As the chart
demonstrates, technical issues were present and the course instructions could be improved. The
learners indicated that the course pace was appropriate and they found it easy to navigate (Figure
1).
21
Figure 1. Usability feedback. Scores were lowest regarding the course being free of technical
issues and ease of navigation.
Technical issues were experienced within the aquarium’s LMS. As participants finished
each module, the indicator bar did not show percentage completed. After each person completed
the course, the indicator showed only partial or no completion percentage. Also, closed
captioning worked on some participants computers and not on others. It was independent of
which browser was used. These issues will need to be remedied with technical support from the
LMS company prior to rollout.
Learner Experience Results
Four questions were asked regarding course feedback- whether it was enjoyable and if
the content added to their learning. The responses fell within the ‘Agree’ and ‘Strongly Agree”
rankings (Figure 2). The results indicated a high level of enjoyment and that the course content
was at an appropriate knowledge level.
22
Figure 2. Learner Experience feedback indicated a high level of learner satisfaction with the course.
Lesson Objectives Results
Four questions were asked that served to assess the learners’ attitudes toward their
learning as a result of course participation and of their attitudes toward practicing preventative
measures in the future. Again, the responses were primarily in the “Agree” and “Strongly Agree”
categories indicating that the course promoted learning and motivation (Figure 3).
Figure 3. Course Objective feedback indicated the high impact on knowledge acquisition, confidence, and
the willingness to adopt the new performance standard.
23
Anecdotal comments from the post-course survey are included (Appendix C). Most of the
comments are positive for content and design. The only negative comments were related to
technical issues which were remedied with course revisions as a result of the functionality tests
and feedback.
Revisions
Based on the functionality assessment, observation notes, and feedback I made the
following revisions to the course:
• reduced the font size and aligned rollover fonts for consistency throughout the course,
• added ‘advanced actions’ for all disease slides in Module 3 so the tabs indicated which
section the learner is viewing,
• changed Module 3 from a branching to a linear format,
• edited the course instructions,
• added indicators for each animal group section in Module 3 so the learner knows their
location in the course, and
• added more diversity of disease symptom images to reduce repetition.
Data Analysis
The expected outcomes of participating in the training course were that upon completion,
there would be an increase in knowledge related to the course objectives as reflected in the
assessment questions. Therefore, a one-tailed t-test was conducted using a directional research
hypothesis (H1: P1>P2) with an accompanying null hypothesis (H0: P1<=P2) with α= 0.05.
24
The charted results suggested that most of the learners scored significantly higher on the
post-knowledge assessment than they did on the pre-knowledge assessment (Figure 4).
Figure 4. Comparison of Pre- and Post-Knowledge Tests. Learners 1 through 8 experienced an increase
in knowledge (Learner 7 scored zero points in the pre-knowledge test).
A paired t-test was conducted between the before and after results of the knowledge tests.
Because the p-value (0.000826268, DF=7) < α (0.05), the null hypothesis was rejected.
Table 3
Pre- and post-assessment scores, t-Test
t-Test: Paired Two Sample for Means
Variable 1 Variable 2
Mean 2.375 5.25
Variance 1.982142857 0.5
Observations 8 8
Pearson Correlation -0.107624401
Hypothesized Mean Difference 0
df 7
t Stat -4.952088328
P(T<=t) one-tail 0.000826168
t Critical one-tail 1.894578605
P(T<=t) two-tail 0.001652336
t Critical two-tail 2.364624252
Note: α = 0.05
Table 3. t-Test of pre- and post-assessment scores
25
Furthermore, the t stat value (4.952) > t critical one-tail value (1.894), which also indicated that
the null hypothesis is rejected. These results indicate that there was a significant difference in
knowledge after taking the course (Table 3).
Data Summary
The effectiveness of the Zoonotic Disease Training course was supported by the data
analysis from the pre- and post-knowledge assessments. The research hypothesis (that learners
gain knowledge by completing the course) was substantiated by the results of the paired t-tests
conducted. A significant difference in knowledge after taking the course was discovered, which
suggested that learning indeed took place. One area of concern arose pertaining to the test
question involving proper reporting steps for injuries. It was pointed out that the aquarium
reporting protocol within the course had conflicting information within the document which
added confusion to that particular assessment question correct answer. The document and
reporting expectations will need revision prior to rollout.
The survey results and observations feedback further supported that zoonotic disease
knowledge increased from taking the course. The higher ratings reported for the increased
confidence of practicing preventative measures is a positive indicator that the learners are more
motivated to follow the new glove-wearing requirement. Furthermore, the self-reported usability
information confirmed that the overall course was user-friendly and enjoyable.
Conclusion
The zoonotic disease training course will serve the purpose of fulfilling the AZA’s
compliance requirements for the Husbandry Division employees and volunteers. The
functionality tests and the course assessment results were good indicators of the potential course
success at meeting the learning and performance objectives. Participants indicated that the course
was enjoyable and that it was informative. Those tested who were employees (two of which were
managers in the Husbandry Division) anecdotally mentioned that it was an improvement over the
current training course. I was pleasantly surprised by the summative evaluation results and the
feedback provided in the post-course surveys.
26
The entire training course including the instructor-led session (not included here as a
product) will be implemented at the Monterey Bay Aquarium in the first quarter of 2018. Further
revisions will be completed post-graduation, to include:
• confirmation of the correct reporting expectations for zoonotic diseases and
animal-related injuries,
• SME confirmation of the disease content information and any need for additional
information,
• additional scenario-based interactive knowledge checks for Module 1,
• clarification of the terminal objective regarding glove-wearing ‘except when
feeding sea otters,
• increasing the timing of text bubbles in the intro video,
• changing one quiz question within the LMS for clarity, and
• augmenting the current design for consistency with increased aquarium branding.
The course will also be customized for other departments within the aquarium based on
the learners’ roles and exposure risks with the animals. There are four other departments that
have requested zoonotic disease training for their employees and volunteers.
My learning curve within Adobe Captivate was impressive. As I gained more confidence,
skills, and awareness of advanced functions I was tempted to revisit what I had developed and to
augment the interactive nature of the course as well as the design. Realizing that this would
affect the timeline for delivery, I accepted the current state of the overall design, navigation, and
interactions knowing that I could make improvements post-graduation.
Awareness of my new instructional design skillset has increased within the Husbandry
Division and across the aquarium. This acknowledgment has resulted in more training design
requests, existing LMS course design questions, and larger training systems design opportunities.
Instructional design principles and practices have now been introduced to the organization
through my efforts with this Capstone Project and by involving multiple stakeholders with the
analysis, design, design, development, implementation, and evaluation phases. A future job
opportunity within the Husbandry Division has also resulted as the aquarium strategically shifts
toward a ‘learning organization’ culture.
27
References
Accreditation standards and related policies. 2016. The Association of Zoos and Aquariums.
Retrieved from https://www.aza.org/assets/2332/aza-accreditation-standards.pdf
Activity Theory. (2007, January 21). Retrieved from https://www.learning-theories.com/activity-
theory.html
Bloom’s Taxonomy: The Affective Domain. (n.d.). Retrieved December 4, 2016, from
http://www.nwlink.com/~%E2%80%89Donclark/hrd/Bloom/affective_domain.html
California Code of Regulations, Title 8, Section 5199.1. Aerosol Transmissible Diseases -
Zoonotic. (n.d.). Retrieved November 9, 2016, from https://www.dir.ca.gov/title8/5199-
1.html
Interview with Jack Phillips on ROI for eLearning – Mindflash Blog. (n.d.). Retrieved December
4, 2016, from https://www.mindflash.com/blog/interview-with-jack-phillips-on-roi-for-
elearning/
Knowles, M. S., Holton III, E. F., & Swanson, R. A. (2014). The adult learner: The definitive
classic in adult education and human resource development. Routledge.
Laberge, M., MacEachen, E., & Calvet, B. (2014). Why are occupational health and safety
training approaches not effective? Understanding young worker learning processes using
an ergonomic lens. Safety Science, 68, 250–257.
https://doi.org/10.1016/j.ssci.2014.04.012
Stolovitch, H., & Keeps, E. (2011). Telling ain't training. American Society for Training and
Development.
Williams, C. J., Scheftel, J. M., Elchos, B. L., Hopkins, S. G., & Levine, J. F. (2013).
Compendium of Measures to Prevent Disease Associated with Animals in Public
Settings, 2013. Journal of the American Veterinary Medical Association, 243(9), 1270–
1288.
Vredenburgh, A. (2002). Organizational safety: Which management practices are most effective
in reducing employee injury rates? Journal of Safety Research, 33, 259–276.
Activity Theory. (2007, January 21). Retrieved from https://www.learning-theories.com/activity-
theory.html
28
Appendix A
Pre-course survey questions
• Age
• Gender
• Are you currently employed as an animal care provider at the Monterey Bay Aquarium?
• Are you currently or previously a volunteer at the Monterey Bay Aquarium?
• Have you previously taken zoonotic disease training at aquarium?
• Have you ever been treated for an animal-related injury?
Post-course survey questions
• Did you have any trouble accessing the course?
• The instructions in the course were easy to follow.
• The course was easy to navigate.
• The pace of the course was appropriate.
• The course was free from technical problems
• The course was enjoyable.
• The course quiz had an appropriate level of difficulty.
• The course content was at an appropriate knowledge level.
• The knowledge checks added to my learning.
• What did you like MOST about the course?
• What did you like LEAST about the course?
• If you could change anything in the course, what would it be?
• I learned something new about zoonotic disease transmission.
• I feel more confident about how to protect myself from zoonotic disease transmission.
• The course increased my willingness to wear gloves to prevent zoonotic disease
infection.
29
Appendix B
Knowledge Assessment questions
• What is the correct definition of zoonotic diseases?
• If you suspect you have a zoonotic disease infection or an animal-related injury, what are
the proper steps for reporting it?
• What are the three most common routes of disease transmission at the aquarium?
• The Husbandry Division requires employees and volunteers to wear gloves when
handling food items except when feeding sea otters.
• Select which circumstances increase your exposure risk to zoonotic disease infection.
• All zoonotic diseases are communicable.
30
Appendix C
Annecdotal post-course survey and observation comments.
What did you like most?
• Pictures of the effects of the diseases.
• Pictures
• It’s very relevant to the job I do
• Dr. Mike, voiceover and visuals
• Great images and voice recording.
• Very informative
• More entertaining than the older version; more specific details
• I learned about zoonotic disease, something I was unaware of.
What did you like least about the course?
• Too much detail about each disease and information about them
• None
• Some slides aren't pretty to look at.
• Just the technical issues.
• Nothing
• Some images. A few were up when the page opened instead of being allowed to
select them.
If you could change anything in the course, what would it be?
• None
• I’d get paid to make Adobe Illustrator illustrations to go along with the photos.
• Nothing seems to need to be changed as far as content. There were a few technical
issues but nothing else.
