Peters HiD Manuscript

1Running Head: THE EVALUATION OF MEMORY LOGS

The Evaluation of Memory Logs on a Brain Injury Unit

Emily S. Peters

Elizabethtown College


The Brain Injury Association of America (BIAA) (2013) estimates that there are 5.3

million Americans who are currently living with a long-term disability as a result of a traumatic

brain injury (TBI). One of the greatest challenges experienced by this population following a

brain injury is memory impairment (Armstrong, McPherson, & Nayar, 2012; Pendleton, 2013).

According to the literature, one of the most effective interventions to treat memory impairment is

the use of an external memory aid (McKerracher, Powell, & Oyebode, 2005). These tools are

utilized in many inpatient rehabilitation settings. One such facility, WellSpan Surgery and

Rehabilitation Hospital (WSRH), expressed the need for a program evaluation to examine the

effectiveness of the use of memory logs by staff on the Brain Injury Unit. This program

evaluation was conducted collaboratively by an external evaluator, an occupational therapy

student from Elizabethtown College, and two on-site occupational therapists at WSRH. The

results lead to a plan for the implementation of recommended changes which will improve the

overall effectiveness of memory log use by staff at WSRH in the future.

Literature Review

This review will begin by providing a brief overview of brain injury etiology and

prevalence in the United States. The subsequent sections will discuss memory function relative

to brain injury, and examine the respective advantages and disadvantages of remedial and

compensatory approaches to treatment of memory impairment. External memory aids will be

introduced as one of the most effective interventions for memory impairments. Three different

implementation protocols for memory notebooks will be discussed in chronological order of their

publication date. WSRH will be introduced as a facility that is currently using memory

notebooks to address the memory impairments of clients receiving treatment in a Brain Injury

Program. This section will conclude with a description of the components of a program


evaluation that was conducted at WSRH to examine the current effectiveness of memory logs by

staff.

Overview of Brain Injury in the United States

According to the National Institute of Health (NIH) (2013), 2.6 million people will

sustain a brain injury each year in America. Brain injuries can be classified as either traumatic or

acquired. A traumatic brain injury (TBI) may occur when there is a direct blow to the head. The

most common causes of a TBI are falls, motor vehicle accidents, sports-related injuries, and

violent assaults. The most at risk groups for a TBI are children between the ages of 0-4 years,

adolescents between the ages of 15-19 years, and older adults over the age of 65 years. In all age

groups, males are more likely than females to sustain a TBI (National Institute of Health, 2013).

In contrast, acquired brain injuries are not induced by trauma. They can occur at any stage in the

lifespan as a result of a cerebrovascular accident (CVA), brain tumor, brain infection, or other

anoxic/hypoxic event (Brain Injury Association of America, 2013).

Both traumatic and acquired brain injuries cause damage to neurons surrounding a given

lesion. Necrosis can result in deficits in motor, cognitive, perceptual, and speech functions.

Function following a brain injury is variable among individuals and depends on the location and

severity of the lesion. One of the most pronounced challenges experienced by this population is

memory impairment (Armstrong, McPherson, & Nayar, 2012; Morris & Reinson, 2010;

Pendleton, 2013; Sandler & Harris, 1992). As many as 80% of clients who sustain a brain injury

will experience a memory impairment as a result of the injury (Barker-Collo & Feigin, 2008).

Memory impairment can have a significant impact on an individual’s functional independence in

everyday, meaningful occupations.


The Impact of Brain Injury on Memory Functions

Following a brain injury, each client is unique in terms of the resulting of a memory

impairment. Location and severity of the brain lesion determine how memory function is

impacted. Memory function can be divided into various subtypes, including prospective and

retrospective memory (Morris & Reinson, 2010). Prospective memory is one’s ability to

remember to do things in the future. For example, prospective memory is applied when one

remembers to engage in a nightly medication regime before going to bed. According to Fleming,

Shum, Strong and Lightbody (2005), prospective memory is the most commonly impaired type

of memory as a result of a TBI due to the vulnerability of the prefrontal lobe as the site of either

coup or contrecoup. Retrospective memory is the ability to remember past events or previously

learned information (Fleming et al., 2005), and is also often impacted following a brain injury.

An example of the use of retrospective memory is when an individual recalls what they ate for

breakfast that morning.

Addressing Memory Impairments in Rehabilitation

One’s daily habits, routines and roles can be disrupted as a result of compromised

memory function (Morris & Reinson, 2010). Memory deficits can significantly limit one’s

functional independence and can lead to frustration and embarrassment. Early in the acute stages

of a brain injury, a rehabilitation team should address memory impairment in treatment in order

to achieve the best possible outcomes for the client (Fleming et al., 2005).

Within rehabilitation, there are two primary approaches that can guide the treatment:

remediation or compensation (Fleming et al., 2005; Pendleton, 2013). A remedial approach to

the treatment of memory impairment focuses on restoring an individual to their prior level of

functioning by re-establishing neural networks that were lost as a result of the brain lesion. This


is accomplished through the use of repetitive mnemonics or memory tasks. Remediation of

memory function has been shown to produce objective improvements on standardized test results

for clients with memory impairment. However, remediation has been unsuccessful in facilitating

the generalization of memory improvements to everyday, purposeful activities for clients

(Fleming et al., 2005).

According to McKerracher, Powell, and Oyebode (2005), a compensatory approach is a

more appropriate treatment for clients with memory impairments. In contrast to remediation, this

approach emphasizes modifying a client’s environment in a way that allows them to adapt to the

memory impairment (Mckerracher, Powell, & Oyebode, 2005). The aim of treatment is to

maximize the client’s abilities without directly addressing the underlying neurological cause of

the memory impairment (Fleming et al., 2005). Treatment may include adding or removing an

object from one’s environment, or modifying an aspect of one’s daily routine to facilitate optimal

memory function. Compensation for memory impairment has been shown to be successful in

promoting generalization of memory improvements to everyday, purposeful activities

(Pendleton, 2013). One of the most commonly-used compensatory interventions is providing the

client with an external memory aid. According to McKerracher, Powell, and Oyebode (2005),

external memory aids (EMAs) have been shown to be the most effective compensatory

intervention for clients with memory impairments.

External Memory Aids: A Means of Compensation

Memory aids come in many forms, including diaries, notebooks, calendars, journals or

activity logs. Memory aids can assist an individual with orienting to their surroundings,

remembering past events, organizing future schedules, and, in some cases, affirming their

progress since the onset of the impairment. According to Pendleton (2013), memory logs can


assist in increasing a client’s self-confidence. Assuming a client has made progress in treatment,

reviewing what has been recorded in the memory aid can affirm accomplishments that have been

made in past therapy sessions. This affirmation of progress is a significant aspect of treatment for

individuals who have sustained a brain injury, considering that they often possess altered self-

concept as a result of the brain injury (Pendelton, 2013). Memory logs are widely utilized in the

rehabilitation setting and have the potential to be highly effective if used correctly (Barker-

Collor & Feigin, 2008).

According to Fleming et al. (2005), a precursor to successful memory aid use by the

client is self-awareness regarding their injury and abilities. This is essential for a client to

understand the purpose and correct use of a memory aid. Additionally, memory aids are most

effective when there is equal buy-in by all members of the interdisciplinary rehabilitation

treatment team (Armstrong, McPherson, & Nayar, 2012). All members of the team should

demonstrate a cohesive understanding of the purpose and value of memory aids. Several studies

have explored the use of memory aids, identified barriers and facilitators to successful memory

aid use, and developed recommendations for their successful implementation. The next section

focuses on methods to train clients and staff on the correct use of memory notebooks

(Armstrong, McPherson, & Nayar, 2012; Donaghy & Williams, 1998; Fleming et al., 2005;

Sohlberg & Mateer, 1989).

Acquisition, application, and adaptation.

Sohlberg and Mateer (1989) are known as the pioneers of the first widely-recognized

memory notebook training protocol. Their protocol consists of three phases: acquisition,

application and adaptation. First, in the acquisition phase, a question-and-answer format is used

to teach clients how to use the notebook. Next, in the application phase, role playing is used to


teach clients how to record information in different sections of the notebook. Lastly, in the

adaptation phase, clients practice recording information in the notebooks in their natural

contexts. Upon its initial development, this protocol was the most widely-applied in brain injury

programs (Sohlberg & Mateer, 1989). However, the original protocol was subsequently criticized

by Donaghy and Williams (1998) as being too lengthy during the acquisition phase, confusing in

terms of the format of the memory notebook, and complex during the training process for those

who have underlying memory impairments.

Drawing on client’s strengths when using a memory notebook.

Donaghy and Williams (1998) suggested the application of a different memory notebook

training protocol as an alternative to that proposed by Sohlberg and Mateer (1989). Their new

protocol was titled the Alberta Hospital Ponoka (AHP) Memory Journal Programme.

According to this program, the three overarching goals of a memory notebook are to

record past and future events, help clients organize log notes, and train clients to use and

maintain the notebook independently. The formatting includes a section titled “Things to Do” for

each day, a section for a daily schedule under which events can be crossed out once they are

complete, and a section for daily notes. This program suggests that training a client how to use

the memory notebook should be done in five phases. Training sessions should be shorter in

duration compared to the original acquisition, application and adaptation protocol, but they

should occur more frequently. A distinguishing feature of this program is the emphasis placed on

drawing on clients’ strengths of immediate recall, procedural memory and old learning. Even

clients with the most severe memory impairment can benefit from drawing on these strengths

(Donaghy & Williams, 1998).


Making it real for the client using a memory notebook.

A study published by Armstrong, McPherson and Nayar (2012) revealed yet another

approach to training clients and staff on the use memory notebooks in an inpatient rehabilitation

setting. The authors collected data through a literature review and semi-structured interviews of

eight occupational therapists who had experience providing treatment to clients with a TBI.

Based on their findings, they identified three overlapping processes that occur during successful

memory notebook training: (a) developing client insight, (b) getting client buy-in, and (c) getting

others on board. This program is tailored to occupational therapists as being the primary

initiators of the memory notebook training in the inpatient rehabilitation setting. The first step of

this program is to develop insight by educating the client on their brain injury. Decreased client

awareness of their memory impairment is one of the major barriers to successful use of a

memory notebook (Fleming et al., 2005). This phase emphasizes the emotional support of the

client through active listening (Armstrong, McPherson, & Nayar, 2012).

Next, getting client buy-in involves including the client in the decision-making process

when formatting the notebook. The client should feel in control and motivated to continue with

training. The last step, getting others on board, involves the occupational therapist recruiting

other members of the healthcare team, as well as caregivers, to assist with the training. Anyone

who is involved with the client should share the same perspective regarding the purpose, value

and expectation of the memory notebook (Armstrong, McPherson, & Nayar, 2012).

All three of these overlapping processes should ideally result in a process called making

it real. During this process, the therapist encourages the client to integrate their use of the

memory notebook into their meaningful, goal-oriented tasks. This encourages the client to

maintain the use of their notebook after discharge from an inpatient rehabilitation setting.


Making it real for the client facilitates generalizability of the memory notebook that supports

independence in everyday life (Armstrong, McPherson, & Nayar, 2012).

Summary.

Training clients and staff on the purpose, value and expected use of memory notebooks is

essential to their effectiveness. As seen in the previous sections, three different protocols have

outlined recommendations that can be applied to assure quality service delivery when utilizing

memory notebooks in a rehabilitation setting. These implementation protocols can serve as

guides for facilities who are utilizing memory notebooks as part of a brain injury program.

WellSpan Surgery and Rehabilitation Hospital (WSRH) located in York, Pennsylvania, is one

such facility that is currently implementing the use of memory logs in a Brain Injury Program.

WellSpan Surgery and Rehabilitation Hospital

WellSpan Surgery and Rehabilitation Hospital functions as a part of the larger integrated

health care system referred to as WellSpan Health. WSRH is a fairly new facility that opened in

April 2012 and that offers orthopedic surgery, neurosurgery, and inpatient rehabilitation services.

Within the rehabilitation department, there is an inpatient Brain Injury Program, which is

described in the subsequent section.

The Brain Injury Program at WSRH is managed by an interdisciplinary team consisting

of occupational therapists, speech therapists, physical therapists, case managers, nurses, a

recreation therapist, and a rehabilitation psychologist. The development of the Brain Injury

Program was, and continues to be, based on the Medical Rehabilitation Standards Manual

(2012). These national standards are mandated by the Commission on Accreditation of

Rehabilitation Standards (CARF), and address criteria for the delivery of quality service from a

variety of specialties within rehabilitation. According to the Medical Rehabilitation Standards


Manual (2012), “A Brain Injury Program is specialized, interdisciplinary, coordinated, and

outcomes focused” (p. 233). The result of a brain injury is highly variable among clients and can

lead to physical, cognitive, psychosocial and behavioral impairments. Ideally, a Brain Injury

Program should strive to address the needs of each client in each of these areas of function

(Brain Injury Program, 2012). One way that WSRH is working to uphold these standards is

through the use of Journey Binders and Memory Logs.

Every client is issued a Journey Binder by the nursing staff within the first few days of

their stay in the inpatient rehabilitation unit. This three-ring binder serves the purpose of

educating the client and caregiver about diagnostic and prognostic information, as well as billing

information. The sections are separated using dividers with tabs that indicate what is included in

each section. Binders are pre-assembled with documents regarding a specific diagnosis. For

example, a client admitted due to a TBI receives a different binder from a client who is admitted

due to a CVA. Documents can be added to the binder as needed throughout the course of

treatment. Journey Binders are intended to serve as an easily accessible resource and should

therefore be with the client at all times. This includes the client taking the Journey Binder to the

therapy gym when they are scheduled to receive treatment.

A client is issued memory log sheets to add to the front of the Journey Binder if the

treatment team feels that they would benefit from this type of intervention. Memory logs are

issued based on the professional judgment of the rehabilitation team. They are used to record the

client’s daily progress in therapy. The goal of the memory log is to optimize memory functions

by assisting the client with their recall of past events and organization of future events. Many

clients who are admitted to the Brain Injury Program are issued a memory log to compensate for

memory loss that has occurred as a result of the brain lesion.


Program Evaluation

Program evaluation is an ongoing process which is necessary in order to best serve clients

who receive rehabilitation services. The objective of a program evaluation is to gather

information with the intention of answering questions that program managers have about a

program (McDavid & Hawthorn, 2006). In spring 2013, the administration of WSRH expressed

concern about the current use of memory logs by staff and they wished to conduct a program

evaluation. The primary objective of this program evaluation was to examine the effectiveness of

the use of memory logs by staff, and to identify barriers and facilitators relative to effective

memory log utilization. A secondary objective was to develop recommendations to improve the

overall effectiveness of memory log use by staff.

Type of program evaluation.

The intention of a formative type of evaluation is to examine program effectiveness in

order to provide feedback for program improvement. In a formative evaluation, it is assumed that

the program should remain in existence and its continuation is not questioned. This is in contrast

to a summative type of evaluation, which is concerned with providing information to make

decisions about a program’s overall worth. In this type of evaluation, a program’s continuation is

in question (McDavid & Hawthorn, 2006). Both types are necessary and essential components to

the program evaluation process. The program evaluation conducted at WSRH was formative

because the effectiveness of memory notebooks was being examined, and the existence of the

Brain Injury Program was not being questioned.


General process of memory log program evaluation.

The structure of this program evaluation followed ten standard steps which guide many

program evaluations (McDavid & Hawthorn, 2006). The questions and their respective answers

relative to this program evaluation at WSRH were as follows:

Who were the client(s) for the evaluation? The clients for this evaluation were the

administration at WSRH, staff providing inpatient rehabilitation services on the

Brain Injury Unit, and the clients consuming these services.

What were the questions and issues driving the evaluation? Administrators at

WSRH were concerned about the current effectiveness of staff utilization of

memory logs. More specifically, they were concerned that memory logs were not

used consistently or properly according to the facility’s protocol.

What resources were available to do the evaluation? An external evaluator (a

senior occupational therapy student at Elizabethtown College) was available to

commit the time in order to collect and analyze the data. On-site staff at WSRH

were also available to commit the time in order to offer insights regarding the

current use of memory logs. Two on-site occupational therapists were available to

serve as supervisors of the evaluator. No funding was necessary to complete this

evaluation.

What has been done previously? No formal program evaluations to assess the

effective use of memory logs have been done thus far in the Brain Injury Program

at WSRH. As previously described, protocols for the use of memory logs with

brain-injured clients can be found in the literature (Armstrong, McPherson, &

Nayar, 2012; Donaghy & Williams, 1998; Fleming et al., 2005; Sohlberg &


Mateer, 1989). This information was used to design data collection tools for the

program evaluation at WSRH.

What was the program all about? As previously described, the Brain Injury

Program was WSRH was and continues to be based on the Medical Rehabilitation

Standards Manual (2012). The Brain Injury Program seeks to deliver services

which address each client’s individualized needs.

What kind of environment did the program operate in and how did that affect the

comparisons available to an evaluator? The Brain Injury Program exists in an

inpatient rehabilitation setting, which is located within a larger hospital that offers

surgery services in addition to rehabilitation. The hospital is fairly new; It opened

in April 2012.

Which research design alternatives were desirable and appropriate? The research

design that was deemed to be most desirable and appropriate in this context was a

qualitative design, as it promotes analysis of the subjective human experience.

Denzin and Lincoln (2000) explain, “Qualitative researchers stress the socially

constructed nature of reality, the intimate relationship between the researcher and

what is studied, and the situational constraints that shape inquiry” (p.8). This

program evaluation sought to understand the in-depth perspectives of staff who

are representative of a variety of disciplines at WSRH, and was conducted so that

the administrators of WSRH could understand the process of staff utilization of

memory logs.

What information sources were available/appropriate, given the evaluation

issues, the program structure and the environment in which the program


operated? The evaluator was unable to gain information from clients and family

members due to liability reasons, therefore access to information was limited.

However, the evaluator was able to collect information from staff at WSRH. The

qualitative design of the evaluation determined the methods of data collection,

which were semi-structured interviews and observations.

Should the program evaluation be undertaken? It was decided that this program

evaluation should be undertaken. Administration at WSRH had expressed a need

for the evaluation, and the necessary resources were available.

Methods of Program Evaluation

This section will discuss the methods of this program evaluation. The first section will

specifically address the qualitative design of the evaluation. The subsequent section will discuss

how the participants were sampled. Next, the two types of data collection will be discussed, as

well as the data analysis procedures. Lastly, confidentiality and ethical considerations will be

addressed.

Program Evaluation Design

This program evaluation was conducted using qualitative methods. An inductive

approach to data collection and analysis was utilized, which meant that specific observations

were used to gain insight into broader themes (McDavid & Hawthorn, 2006). An assumption of a

qualitative design is that the world consists of multiple realities. In this program evaluation,

multiple staff were interviewed and observed in order to gain a broad understanding of varying

perspectives regarding memory log utilization. Data collection and analysis were descriptive and

interpretive, which are also characteristics of qualitative designs (Carter, 2011).


Participants

The target sample size was eight to ten participants. The goal was to include at least two

staff from each of the following disciplines: occupational therapy (OT), physical therapy (PT),

speech therapy (SLP), and nursing. The plan was for eligible staff to be only those who treat

clients on the Brain Injury Unit. Staff who treat clients on the General Medicine or Surgery Units

were intended to be excluded from the sampling. The goal was to interview up to two staff from

each discipline, and to observe a total of four staff from different disciplines. Participants were

chosen via convenience sampling by the on-site supervisors.

Data Collection

Data collection occurred from November 2013 to April 2014. The two means of data

collection were semi-structured interviews and observations. Both were guided by instruments

developed collaboratively by the evaluator and the on-site supervisors. The evaluator suggested

content that should be included in the data collection instruments based upon characteristics

deemed to be important components of memory log implementation protocols as stated in the

literature (Armstrong, McPherson, & Nayar, 2012; Donaghy & Williams, 1998; Fleming et al.,

2005; Sohlberg & Mateer, 1989). Client and staff training regarding the use of memory logs, and

physical formatting of memory logs, are two examples of topics found in the literature that were

included in the data collection instruments. The on-site supervisors suggested content to be

included in the data collection instruments based on their perception of current memory log use

by staff.

Interviews were conducted using a guide (Appendix A), which was developed

collaboratively in advance by the evaluator and the on-site supervisors. Interviews addressed

staff perspectives’ regarding the purpose, expected use, and current effectiveness of memory log


use at WSRH. Each participant engaged in a face-to-face interview which lasted approximately

30 minutes. The evaluator wrote brief notes during each interview, as they were not audiotaped.

Field notes were written immediately after each interview, which detailed the participant’s

responses to interview questions and nonverbal body language during the interview. The

interviews took place in a conference room located near the therapy gym at WSRH. The

evaluator and the interviewee sat at a medium-sized square table directly facing one another. The

evaluator and the interviewee were the only people present in the room during the interview.

Observations of staff during treatment sessions with clients were recorded using a

checklist (Appendix B), which was developed collaboratively in advance by the evaluator and

the on-site supervisors. The purpose of the observations was to gain greater insight regarding the

location of a memory logs throughout a treatment session, and the frequency of memory log

utilization. Each observation lasted approximately one hour. Field notes were written

immediately after each observation session, which further detailed the participants’ responses

and nonverbal body language. The opportunity to observe staff was determined by practicality.

Factors such as time constraints for the evaluator, the availability of the staff who could be

observed, and at what time the observation could occur.

Data Analysis

Thematic analysis using a constant comparative approach was used to identify emerging

themes across participants (McDavid & Hawthorn, 2006). The evaluator closely examined notes

from the interviews and observations, as well as the observation checklists, and identified

recurring themes across the data sources. To assist in the development of the themes, frequency

charts were used to organize the data. Data trustworthiness was enhanced using member

checking via email follow-up with participants, as well as triangulation across data sources.


Confidentiality

Participant pseudonyms were used throughout this evaluation to protect staff

confidentiality. All participants were informed that the data recorded during observations and

interviews would be de-identified. Permission was granted by the administration of WSRH to

include the name of the facility in the results. Additionally, permission was granted by the two

on-site supervisors to include their names and credentials in the acknowledgement section of this

document, as well as on the associated poster.

Ethical Considerations

An application was submitted to The Institutional Review Board at Elizabethtown

College on September 25, 2013 to determine if the program evaluation met the criteria for

research involving human subject protections. The Board deemed that this project was not

research.

Results of This Program Evaluation

This section begins by describing the participants and the process of data collection.

Next, four themes are explained that were identified as a result of the data analysis.

Participants

The sample included eight participants who were representative of staff from a variety of

disciplines working in the inpatient rehabilitation setting at WSRH. Pseudonyms have been used

to protect staff confidentiality. The participants treated clients on the Brain Injury Unit, General

Medicine/Surgery, or both. As seen in Table 1, only two participants worked on the Brain Injury

Unit full-time. This was not in alignment with the original inclusion criteria, which stated that

only staff who worked on the Brain Injury Unit would be part of the sample. However, the


number of staff who worked only on the Brain Injury Unit was limited. The evaluator and the on-

site supervisors agreed to broaden the sample in order to gain greater breadth of information.

Table 1

Participants

Participant Discipline Unit

David PT Brain Injury

Brad PT General Medicine/Surgery

Lori OT General Medicine/Surgery

Denise OT General Medicine/Surgery

Lisa SLP Brain Injury

Mackenzie SLP General Medicine/Surgery

Heather RT Brain Injury and General Medicine/Surgery

Amanda LPN Brain Injury and General Medicine/Surgery

Data Collection

Every participant engaged in one face-to-face interview. Participants varied in the

number of times they were observed due to practical reasons. It is important to note that two of

the participants, David and Lisa, worked full-time on the Brain Injury Unit and were observed on

more occasions compared to the other participants. This approach enhanced the applicability of

findings specifically to the Brain Injury Unit.


Table 2

Data Collection Methods

Participant Discipline Number of Observations Number of Interviews

David PT Twice Once

Brad PT None Once

Lori OT Once Once

Denise OT None Once

Lisa SLP Three times Once

Mackenzie SLP None Once

Heather RT Twice Once

Amanda LPN None Once

Themes

As a result of data analysis, four themes were identified regarding the current utilization

of memory logs by staff at WSRH. The themes illustrate similarities and differences among staff

opinions regarding how and when memory logs should be used, as well as the ideal physical

format of memory logs.

Participants vary in their beliefs that memory logs should be issued to all clients versus

only those clients with a memory impairment. There was discrepancy among the staff in terms of

the underlying purpose of the memory logs. Some participants conveyed that memory logs are a

tool to assist only those clients who have a marked memory impairment. Denise stated, “They

[memory logs] are not always applicable to the patient.” The evaluator observed this belief

during an observation session with Denise. The client who was receiving treatment from Denise

did not have a memory log. As described previously, Denise works on the General


Medicine/Surgery Unit full-time. She described during her interview that she treats primarily

clients who have an orthopedic diagnosis, and most of them do not have a memory impairment.

Therefore, she feels that memory logs are not necessary or appropriate for these clients.

On the contrary, other participants conveyed that memory logs can serve a broader

purpose. Mackenzie, a SLP, stated, “They [memory logs] can be used to log [all patient’s]

progress and increase their insight.” Lisa, also a speech therapist, conveyed the same belief

during her interview when she stated, “The memory log a serves a purpose greater than

reminding patients with a memory impairment of their to-do schedule. It can be a means of

organizing and reminding patients of their progress.” It is evident that staff have different beliefs

regarding the purpose and ultimate goal of memory logs.

The physical format of the memory log is not conducive to successful implementation.

The evaluator observed that memory log pages are placed in the very front of the client’s Journey

Binder. They are not indicated by a divider tab or label. The memory log sheets do not stand out

and grab the client’s attention. The sheets are organized so that each page provides blank rows

for every hour of the day that the client can use to record their daily therapy activities. There are

rows for 11 hours on each page. Some clients record activities completed outside of therapy

treatment sessions, but Brad revealed during his interview that most clients record only activities

that are completed during therapy treatment sessions. Brad proceeded to explain that this

typically results in unused space on the memory log sheet at the end of the day because each

client receives only three hours of therapy a day.

Many of the participants agreed that both the placement within the Journey Binder and

the layout of the memory log pages are not supportive to effective implementation of memory

logs. Mackenzie stated, “Memory log sheets need to stand out more,” and Brad stated, “The


section for memory log sheets needs to be designated by a colorful divider tab.” These

statements illustrate that the staff perceived that the current physical format of the memory log

interferes with its successful implementation.

There is a lack of staff awareness regarding the expectations of memory logs and staff

roles in the implementation of memory logs. David, a PT, stated, “PTs don’t address cognition,”

in response to the evaluator questioning the location of the Journey Binder which was not present

in the therapy gym during an observation session. As stated previously, Journey Binders are

intended to be with clients at all times. When David’s client came to the gym without their

Journey Binder, David shrugged his shoulders and did not try and find the memory log. The

evaluator learned during the observation sessions that Journey Binders were not in the client’s

possession consistently. For example, sometimes the Journey Binders were left in the client’s

room when they went to the therapy gym for treatment, and other times they would take the

memory log to the therapy gym. This lead to memory logs being used at inconsistent times by

different staff members.

The evaluator also learned from participant interviews that there is inconsistency

regarding who issues the memory logs. They are sometimes issued by SLPs, and other times by

OTs, and this variability in the absence of policy results in memory logs not being consistently

issued. Heather stated, “The speech therapist initiates the memory log within the first week of the

patient’s stay,” and David stated, “Memory logs are issued by the speech therapist or the

occupational therapist.” Both of these statements, made during the participants’ interviews,

illustrate contrasting beliefs regarding staff roles and responsibilities in memory log

implementation. During the observation sessions, the evaluator did not witness any memory logs

being issued.


Some of the participants believe that memory logs are currently being used ineffectively.

Six out of eight participants made statements that support this theme during their individual

interviews. Denise stated, “They [memory logs] are great in theory, but they aren’t working

here.” Denise proceeded to explain that she believes that memory logs are an effective

compensatory strategy for memory impairment, but lack of staff awareness regarding the

memory logs causes them to be ineffective at WSRH. Lori stated, “Memory logs are being used

inconsistently,” and David stated, “They are effective, but they are underutilized.” This theme

was also supported by data which was gathered during observations. For example, the location

where blank memory logs were stored was inconvenient. Participants conveyed during

interviews that the inconvenient location where memory log pages were stored made them less

likely to retrieve memory log pages when one was needed for a client.

Discussion

This section begins by interpreting the results which were described in the previous

section. These interpretations are followed by recommendations that may improve the overall

effectiveness of memory log utilization by staff at WSRH. The changes that WSRH wishes to

implement are addressed. Future program evaluation efforts, limitations, and implications for

practice conclude this section.

Interpretation of Results

Memory logs are currently being used ineffectively by staff at WSRH. Overall, there is a

lack of staff buy-in regarding memory logs, which is acting as a barrier to successful memory log

implementation. An additional barrier is the evident lack of policy to clearly define staff

expectations in memory log implementation. There is no formalized explanation of specific staff

roles in memory log implementation. This results in interdisciplinary discrepancy regarding who


is responsible for issuing memory logs, and which clients should be receiving memory logs.

These varying perspectives among staff are causing inconsistent and ineffective memory log

utilization. This is in alignment with the literature, which states that discrepancy of staff roles

within a program protocol can contribute to a lack of staff buy-in, and further decrease the

likelihood that a program will be effective (Armstrong, McPherson & Nayar, 2012).

Lisa and Mackenzie, both speech therapists, conveyed that they play an instrumental role

in issuing the memory logs, and they also conveyed that they believe that memory logs have a

broad purpose that can benefit all clients. This suggests that speech therapists are the staff with

the most buy-in and are therefore most likely to consistently utilize memory logs. David’s

statement, “PTs don’t address cognition,” implies that he does not believe that it is within his

scope of practice to address memory function. While it is most likely true that he does not have

specific treatment goals focusing on cognition, WSRH expects that all staff should be actively

aware of client’s cognitive status and support the goals of other disciplines. According to

protocol for the Brain Injury Program at WSRH, memory log utilization is not intended to be the

specific responsibility of any particular discipline, but rather to be universally utilized by all

disciplines. Armstrong, McPherson, and Nayar (2012) propose that “getting others [staff and

family] on board” is an essential component to successful memory log implementation.

Another barrier is the inconvenience of the location where blank memory log pages are

stored. Multiple staff reported that it is out of their way to go to the conference room, which is

not in close proximity to the therapy gym, in order to retrieve blank memory log pages when

necessary. In addition to this barrier, another contributing factor to the ineffectiveness of

memory logs is the placement and organization of memory log pages within the Journey Binder.

Memory log pages are not clearly labeled, nor do they capture the client’s attention with brightly


colored labels. These barriers suggest that developing potential recommendations for changing

the memory log protocol may improve the overall effectiveness of memory log utilization by

staff at WSRH.

Recommendations

Based on the results of the data analysis, the evaluator formulated a list of

recommendations that may improve the effectiveness of memory log use by staff at WSRH.

Additionally, if explicit policies and procedures would be made clear to all staff, this could be

used as a basis for further training and program evaluation at WSRH. The recommendations

made by the evaluator were:

Move the blank memory log sheets to a more accessible location within the

therapy gym.

Modify the physical format of the memory log pages.

Initiate a staff in-service to: (a) establish a clear and detailed policy for memory

log utilization, (b) raise awareness about the unique roles and expectations of each

discipline in memory log utilization, (c) convey to staff that memory logs can be

beneficial for all clients.

Change the label of the section in the Journey Binder from “memory log” to

“activity log.”

Moving the blank memory log pages to a more convenient location would increase the

likelihood that staff would retrieve them as necessary. This would further increase the likelihood

that memory logs would be utilized by staff.

As described in the Results of This Program Evaluation section, many staff reported that

the current placement and organization of memory log pages within the Journey Binder are not


conducive to effective implementation. Modifying the physical format of memory log pages in a

way that captures the attention of both clients and staff would increase the likelihood that they

would be used consistently. According to Sandler & Harris (1991), memory logs should be

brightly colored and include the patient’s name in large, easy-to-read letters. The evaluator

recommended that the memory log pages should either be printed on colored paper, rather than

white paper, or they should be indicated in the binder using a colorful divider tab.

The evaluator suggested that memory logs should be issued to all clients at WSRH, rather

than just to clients who have a marked memory impairment. The label “memory log” should shift

to “activity log” to illustrate that these tools can benefit all clients. According to Pendleton

(2013), memory logs can serve as a tool to assist in increasing a client’s self-confidence. When a

client reviews the activities that they have accomplished in a past therapy session, the review

may affirm the progress they’ve made as they compare this to their current level of performance.

Additionally, if every client received a memory log, the possibility of forgetting to issue a

memory log would be reduced or potentially eliminated because there would no longer be a

question about who should receive one. In order to accomplish this, the evaluator suggested that

memory log pages should be routinely included in all Journey Binders.

Lastly, the evaluator recommended that administrators at WSRH could initiate a staff in-

service to: (a) establish a clear and detailed policy for memory log utilization, (b) raise awareness

about the unique roles and expectations of each discipline in memory log utilization, (c) convey

to staff that memory logs can be beneficial for all clients. The purpose of this in-service would be

to increase staff buy-in by emphasizing the unique and specific role of each discipline.

Plans for Implementation


The evaluator conveyed the recommendations with the on-site supervisors at WSRH,

who proceeded to share these ideas with a Journey Binder work committee consisting of staff

from various disciplines. After discussing the list of recommendations, the committee agreed on

changes which they felt would be effective if implemented. The changes will be:

“Memory Logs” will be referred to as “Activity logs.”

Every client will receive a memory log.

Activity Logs will be identified in the Journey Binders by a colorful tab that reads “My

Activity Log.”

Therapy staff will be educated on their individual roles in the use of Activity Logs during

a weekly staff meeting.

Future Program Evaluation

The administrators of WSRH plan to educate the staff about their proper roles in the use

of Activity Logs during a weekly staff meeting. However, the following questions remain:

How will they be educated?

How will the unique roles of each discipline be implemented in the new Activity Log

protocol?

In order for the planned changes to be most effective, these questions should be thoughtfully

considered by the administrators at WSRH. Additionally, continued program evaluation is

necessary to measure the determined outcomes of the implemented changes. Further evaluation

should assess if the implemented changes resulted in more effective utilization of Activity Logs

by staff at WSRH. Questions to consider in further program evaluation include, but are not

limited to:


Are Activity Logs used more consistently among staff from all disciplines in

comparison to the previous system of memory logs?

Is the colorful divider tab that reads “My Activity Log” successful in capturing the

attention of both clients and staff?

Program Evaluation Limitations

The first limitation of this program evaluation was the small sample size. The participants

represented only 26% of the full-time, therapy staff, which limited the generalization of results to

all staff members. A second limitation was the method of sampling. The on-site supervisors

chose which staff would be involved, which limited the credibility of the results. The supervisors

may have held biases toward staff which influenced their interpretation of the results. A third

limitation was the lack of frequent contact between the evaluator and the participants. This

limited the opportunity to collect data via face-to-face interviews and observations. This was due

to practical reasons, such as scheduling conflicts and the distance between Elizabethtown

College and WSRH.

Implications for Practice

These results illustrate the importance of program evaluation in the clinical environment.

Without the program evaluation at WSRH, the concerns regarding memory log utilization by

staff may not have been recognized by the administration, and changes for improvement may not

have been developed and implemented. Program evaluation is an essential component of

ensuring quality delivery of rehabilitation services. The results also illustrate that greater,

facility-wide problems regarding productivity demands, space planning, and policy development,

can have an impact service delivery. At WSRH, inadequate policy development was having an

adverse impact on successful memory log utilization by staff. Additionally, inconvenient


placement of blank memory log sheets decreased the likelihood that memory logs would be

utilized. There is a need for administration in the clinical environment to assess these problems

in order to assure quality service delivery for clients.

Occupational therapists will often directly address cognition in the clinical environment.

They can apply these findings to the planning and implementing of external memory aids.

Occupational therapists should consider the therapeutic benefits of external memory aids not

only for those clients who have a memory impairment, but for all clients. They should also

understand how their responsibilities in memory log implementation at their particular setting

differ from the responsibilities of speech therapists and physical therapists. Occupational

therapists can advocate for a cohesive interdisciplinary approach to the implementation of

external memory aids by increasing staff buy-in regarding the therapeutic benefits of these

compensatory tools.

Conclusion

Multiple barriers relative to effective memory log use by staff at WSRH were identified,

including lack of staff awareness regarding roles in memory log implementation, and the

expected use of memory logs. The analysis of the results suggest that implementing changes to

the current protocol will improve the overall effectiveness of memory log use. In response to

recommendations for improvement that were developed by the evaluator, the administrators at

WSRH developed specific plans to implement several of the recommendations. Ongoing

program evaluation efforts will be necessary to ensure that the use of external memory aids at

WSRH will remain an effective component of their rehabilitation services.


Acknowledgements

I would like to recognize individuals who were fundamental to the success of this program

evaluation:

Alicia Fry, M.H.A., OTR/L, WellSpan Surgery and Rehabilitation Hospital.

Dr. Linda M. Leimbach, Sc.D., OTR/L, C/NDT, Lecturer, Elizabethtown College.

Megan Dean, M.S., OTR/L, WellSpan Surgery and Rehabilitation Hospital.

Staff participants at WSRH


References

Armstrong, J., McPerson, K., & Nayar, S. (2012). External memory aid training after traumatic

brain injury: ‘Making it real.’ British Journal of Occupational Therapy, 75, 541-548.

Barker-Collo, S., & Feigin, V. L. (2008). Memory deficit after traumatic brain injury: How big is

the problem in New Zealand and what management strategies are available? The New

Zealand Medical Journal, 121(1268), 1-7.

Commission of Accreditation of Rehabilitation (2012). Brain injury program. Medical

Rehabilitation Standards Manual (pp. 233-239).

Brain Injury Association of America. (2013). Brain injury statistics. Retrieved from

http://www.biausa.org/glossary.htm.

CARF International. (2014). Medical Rehabilitation. Retrieved from

http://www.carf.org/Programs/Medical/.

Carter, R.E., Lubinsky, J., & Domholdt, E. (2011). Qualitative research. Rehabilitation research:

Principles and applications (4th ed.) (pp. 157-173). St. Louis: Elsevier Saunders.

Denzin, N. K., & Lincoln, Y. S. (Eds.). (2000). The discipline and practice of qualitative

research. Handbook of qualitative research (2nd ed.). (pp. 1-36). Thousand Oaks, CA:

Sage.

Donaghy, S., & Williams, W. (1998). A new protocol for training severely impaired patients in

the usage of memory journals. Brain Injury, 12, 1061-1076.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative

approaches and practical guidelines (3rd ed.). Boston, MA: Pearson Education, Inc.

http://www.carf.org/Programs/Medical/

http://www.biausa.org/glossary.htm


Fleming, J. M., Shum, D., Strong, J., & Lightbody, S. (2005). Prospective memory rehabilitation

for adults with traumatic brain injury: A compensatory training program. Brain Injury,

19(1), 1-10.

Greenaway, M. C., Duncan, N. L., & Smith, G. E. (2012). The memory support system for mild

cognitive impairment: Randomized trial of a cognitive rehabilitation intervention.

International Journal of Geriatric Psychiatry, 28, 402-409.

Jacobs, K., & McCormack, G. L. (Eds.) (2011). The occupational therapy manager (5th ed).

Bethesda, MD: American Occupational Therapy Association.

Lee, S. S., Powell, N. J., & Esdaile, S. (2001). A functional model of cognitive rehabilitation in

occupational therapy. Canadian Journal of Occupational Therapy, 68, 41-50.

McDavid, J. C., & Hawthorn, L. R. L. (2006). Program evaluation and performance

measurement: An introduction to practice. Thousand Oaks, CA.

McKerracher, G., Powell, T., & Oyebode, J. (2005). A single case experiment design comparing

two memory notebook formats for a man with memory problems caused by traumatic

brain injury. Neuropsychological Rehabilitation, 15, 115-128.

Morris, K., & Reinson, C. (2010). A systematic review of the use of electronic memory aids by

adults with brain injury. Technology Special Interest Section Quarterly, 20(1), 1-3.

National Institute of Health. (2013). How many people are affected by or at risk for TBI?

Retrieved from https://www.nichd.nih.gov/health/topics/tbi/conditioninfo/Pages/affected

risk.aspx .

O’Callaghan, C., Powell, T., & Oyebode, J. (2006). An exploration of the experience of gaining

awareness of deficit in people who have suffered a traumatic brain injury.

Neuropsychological Rehabilitation, 16(5), 579-593. doi: 10.1080/09602010500368834.

https://www.nichd.nih.gov/health/topics/tbi/conditioninfo/Pages/affected%09risk.aspx

https://www.nichd.nih.gov/health/topics/tbi/conditioninfo/Pages/affected%09risk.aspx


Pendleton, H. M. H., & Schultz-Krohn, W. (2012). Evaluation and treatment of limited

occupational performance secondary to cognitive dysfunction. Pedretti's occupational

therapy: Practice skills for physical dysfunction (7th ed) (pp.648-677). St. Louis, MO:

Mosby/Elsevier.

Sandler, A. B., & Harris, J. L. (1992). Use of external memory aids with a head-injured patient.

American Journal of Occupational Therapy, 46, 163-166.

Sohlberg, M. M., & Mateer, C. A. (1989). Introduction to cognitive rehabilitation. New York,

NY: The Guilford Press.

Wilson, B.A., Emslie, H., Quirk, K., Evans, J., & Watson, P. (2005). A randomized control trial

to evaluate a paging system for people with traumatic brain injury. Brain Injury, 19(11),

891-894.


Appendix A

Interview Guide1. As a (PT, OT, SLP, recreational therapist, or nurse) how do you work with patients who

have memory impairments? 2. (Depending on first question) What is your level of familiarity with memory notebooks?

a. In your opinion, what is the purpose of a memory notebook? b. Are they an effective form of intervention for memory impairment? Why or why

not?c. How do you know if a patient has a memory notebook? d. If they do not have a memory notebook, how do you obtain one? Who issues

them? e. Where is a patient’s memory notebook located?

3. Do you ever provide patient education regarding the memory log? If so, how do you go about doing this?

a. Family/caregiver education? b. Upon discharge, should the memory notebook be left in the rehabilitation hospital

or should the patient take it along with them? Do you encourage them to use it at home?

4. How do you think the current system of memory books is working in this facility?a. What do you like about the current method of using memory notebooks?b. What barriers do you find when using the memory notebooks? c. Should memory notebooks be the standard intervention used for all patients? Or

just those with a memory impairment?d. What are ways that our approach could be more customized to meet the needs of

clients & families, without sacrificing efficiency & effectiveness?


Appendix B

Observation ChecklistThis checklist will be used to record information regarding the utilization of memory logs by staff during a treatment session. The “comments” section should be used to include additional details about the checked boxes. Date: _______________________________________Time at beginning of observation: _______________________Time at end of observation: ___________________________Total amount of time spent observing staff: ___________________Staff discipline observed:

Physical TherapyOccupational TherapySpeech TherapyNursingPsychologyRecreation Therapy

Location of staff observationPatient’s roomRehabilitation gymSLP officeOutsideDining RoomOther

Purpose of staff interaction with patient: Initial EvaluationIntervention sessionCareDischarge

If intervention, specify what type: ADLs:

Therapeutic exercises:

Patient education:

Functional mobility/gait/ambulation/stairs:


Transfers:

Homemaking tasks:

Cognitive tasks:

Other:

Section 1: Format of memory notebook Yes No ----------------------------------------------------------------------------------------------

Outside of notebook is brightly coloredPatient’s name is written in large, easy-to-see letters on cover of notebookNotebook includes daily scheduleNotebook includes orientation information (day of week, date of month, season, etc.)Notebook includes monthly calendarNotebook includes a section to record notesNotebook has a pen attached There are extra log sheets in the back of the notebook

Comments:

Section 2: Location of memory notebookBeginning of observationYes No Location

Located on bedside tableOn wheelchairIn a drawerIn a file holder mounted to the wall or doorNot in the roomOther

Moved for the first timeYes No Location

Located on bedside tableOn wheelchairIn a drawerIn a file holder mounted to the wall or door


Not in the roomOther

Moved for the second timeYes No Location


Moved for the third timeYes No Location


Conclusion of observationYes No Location


Comments:

Section 3: Staff cuing and outcomeType of cue & frequency Successful Unsuccessful

No cue givenVerbal cue 1x for memory notebookVerbal cue 2x for memory notebookVerbal cue 3x for memory notebookVerbal cue > 3x for memory notebookVerbal cue 1x for other memory aidVerbal cue 2x for other memory aidVerbal cue 3x for other memory aid


Verbal cue > 3x for other memory aidPhysical assistance 1x for memory notebookPhysical assistance 2x for memory notebookPhysical assistance 3x for memory notebookPhysical assistance for other memory aid

Comments:

Section 5: Education Yes No ----------------------------------------------------------------------------------------------

Staff introduced memory notebook to the patient for the first timeStaff introduced memory notebook to the family for the first timeStaff verbally expressed the purpose of the memory notebook to the patient [not for the first time]Staff verbally expressed the purpose of the memory notebook to the family [not for the first time][If being discharged] staff did not tell patient to take memory notebook along with them[If being discharged] staff did tell patient to take memory notebook along with them

Comments: