Upload
scott-linja
View
228
Download
0
Embed Size (px)
Citation preview
8/2/2019 In Design Group 3 White Paper
1/19
To: Christa TestonCc: Melissa ClarkFrom: Brian Bowles,Dylan Brown, Scott Linja, Brian Lockner, Maxine Major
Infographics Effectiveness
White Paper
Introduction
Wildfire danger is a prevalent concern in the Northwestern United States. Currently land managers
still do not have enough resources to manage the increasing severity of wildfires and costs
associated with wildfire damage and prevention. Melissa Clark from the University of Idahos
Department of Natural Resources Department will be conducting a study on how to engage
residents of the Northwest in supporting legislation to help prevent and treat wildfires, and to take
personal action to minimize the danger of wildfires.
Our study was a pilot study to assist Melissa Clark by testing the effectiveness of a set of
infographics, one gains-framed and one loss-framed infographic, to spread knowledge about the
dangers of wildfires and influence the views of the participants viewing either infographic toward
supporting wildfire control methods. The results from our study will allow us to provide Ms. Clark
with data on which infographic tends to influence a greater change in participants responses, and
to provide feedback on how to maximize the effectiveness of either infographic.
This white paper outlines our study to test the effectiveness of these infographics, and to determineany areas in which the effectiveness of these infographics might be improved. The infographics weused were developed by a technical writing class at the University of Idaho during the Springsemester 2012.
Gains-framed and loss framed infographic versions of the infographic were presented to differentgroups of people. We conducted knowledge and opinion tests before and after participants viewedthe infographics, and the answers to the questions were interpreted to evaluate the participantsknowledge of wildfires and prevention. We also attempted to evaluate how either viewing a gains-framed or loss-framed infographic may have influenced participants to change their opinion aboutthe dangers of wildfires, and helped influence participants toward taking action to preventwildfires.
We aimed to discover the following through this study:
Is either infographic successful at increasing the participants wildfire preventionknowledge?
Does either infographic successfully influence the participants views on wildfires? If either or both of the above is true, is one of the infographics more successful than the
other at this?
8/2/2019 In Design Group 3 White Paper
2/19
In order to determine whether or not viewing an infographic was effective, we evaluated howmuch the individual participants gained knowledge and if their responses indicated a changedperception of fire danger and willingness to participate in wildfire prevention measures.
This study was not designed to tell us how either infographic could be improved to affect
the participants views, or how to convey more knowledge to the participant. This studywas designed to only tell us which infographic, gains-based or loss-based, was moreeffective to convey information, and if participants viewing either infographic were morelikely to take action.
Methods
Our group used three separate tests in order to determine how effective both the gains-framed and
loss-framed infographics were on participants from different demographics. The participants were
selected based upon proximity to the various group members administering the separate tests. The
three groups included a random sample of 15 FarmHouse fraternity members (Tests administeredby S. Linja), 10 Argonaut staff members (Tests adminstered by D. Brown) and 6 Moscow residents
(Tests administered by B. Lockner). A total of 31 people participated in this test.
The tests were administered in this order:
Pre-treatment:
1. Interview
2. Knowledge Quiz
3. Survey
Treatment:
4. Infographic
Post-treatment
5. Knowledge Quiz
6. Survey
Interview (see Appendix B)
A fill-in-the-blank interview was used to examine independent variables within the various test
groups. The variables included gender, major, hometown, home state, rural/urban community,
political affiliation, and previous wildfire experience. The data collected would be analyzed to
determine significant trends between certain demographics compared to the results of the survey.
Knowledge Quiz (see Appendix C)After completing the survey, the participants were given a six question knowledge quiz prior to
viewing the infographic. The knowledge quiz questions were relevant to understanding wildfire
dangers and related information that could be answered by a cursory viewing of either infographic.
Each participant completed the knowledge test, and after treatment, the participant took the
knowledge test again. The responses were analyzed to determine the participants level of wildfire
8/2/2019 In Design Group 3 White Paper
3/19
knowledge prior to viewing the infographic, and how much their amount of wildfire knowledge
changed after treatment.
Survey
After completing the interview, each test group was given a survey that examined each
participant's perceptions and beliefs pertaining to wildfires and prevention. The survey, formulated
by Melissa Clark, investigated the participants existing beliefs/perceptions prior to an examination
of either infographic. After the infographics had been examined during the treatment, each group
was given the survey once more, and results were analyzed to determine if the participants views
had changed.
Treatment (see Appendix A)
After taking all three pre-treatment tests, participants were divided randomly into two groups forthe treatment session. For one minute, each group was allowed to view one of the twoinfographics, either the gains-based infographic, or the loss-based infographic. After one minute,
the infographics were removed from the participants sight, and they proceeded to take the post-treatment tests.
The responses from all three methods were compiled and analyzed to determine if there was a
marked difference between pre-treatment and post-treatment responses.
Results
Note: After testing had completed, it was discovered that it had not been noted which participants
from the Argonaut Staff group had viewed either the gains-based or loss-based infographic. Sinceeach participant did complete the treatment according to the test plan, their overall scores were
included in overall totals, but for the purpose of segregating results based on type of infographic
viewed, their results were excluded entirely.
Knowledge Test Results (See Appendix C)
Prior to viewing the infographics, participants overall scored 48.9% on the knowledge quiz, and
after treatment, overall scores increased by 31.7%. The range of pre-treatment knowledge test
scores differed among the three demographics by 15%, but post-treatment, the range narrowed to a
4.4% difference in scores. (Figure 1)
The group with the lowest pre-treatment scores was the FarmHouse Fraternity, and this group hadthe highest post-treatment increase in test scores. In contrast, the Moscow Residents group startedwith the highest pre-treatment scores, and their post-treatment scores indicated the lowest increasein knowledge gained of all three groups. The Argonaut staff had the highest overall scores at theend of testing. (Figure 1)
8/2/2019 In Design Group 3 White Paper
4/19
Figure 1: Knowledge Test Scores by Demographic
Although recipients of either infographic were selected at random, participants who would receivethe loss-based infographic did worse on the pre-treatment knowledge test than participants whowould receive the gains-based infographic. This is why the difference in scores was higher afterviewing the infographic, despite receiving overall lower post-treatment scores. (Figure 2)
Figure 2: Knowledge Test Scores by Infographic Viewed
Knowledge Test AnalysisThe Moscow Residents demographic is expected to be the most closely related to the actual
intended audience for these infographics, and the results that the infographics had on the Moscow
Residents represent the expected effectiveness of these infographics. Because Moscow residents
had the highest pre-treatment knowledge scores, it could be hypothesized that the average reader of
the infographic may already have a high level of wildfire knowledge, and that the majority of
8/2/2019 In Design Group 3 White Paper
5/19
readers may not have a knowledge increase comparable to the FarmHouse and Argonaut
demographics.
However, the average audience is still expected to gain approximately 30% more knowledge of the
subject matter by viewing either infographic. This may indicate that these infographics could be
useful to educate residents of the Northern Rockies in Idaho and Montana.
The gains-framed infographic readers saw a knowledge gain of 30% while the loss-framed
infographic readers a 33% knowledge gain. Due to the initial lower score of the loss-gains pre-
treatment test, this 3% discrepancy may be considered insignificant when compared to the average
knowledge gain of either infographic. (Figure 2)
All three demographics had significant increases in knowledge. The Farmhouse fraternity scored
low on the pre-treatment knowledge test, and the Moscow Residents scored highest out of all three
groups. After treatment, all three demographics scored significantly higher than their pre-treatmentscores, and the Farmhouse fraternity scores were nearly equal to those of the Moscow Residents.
This indicates that these infographics educated different demographics equally well. (Figure 1)
Knowledge Test Discussion
This knowledge test was simple, brief, and tested participants short term memory to recall wildfire
data successfully. This did not test the longevity of participants memory, and their ability to recall
important information at a later date, such as would be useful in a real-world setting. In addition,
while the infographics gave the participants information, it could have been better developed tohelp participants to recall information reflecting the importance of wildfires and on actions that
individuals can take to prevent wildfires.
Survey Results (See Appendix D)
The survey given to participants was divided into three sections. For the purpose of this study, in
order to evaluate the infographics ability to affect the views of the participants, we focused on the
results of Question #3 only. Question #3 asked participants to rate the perceived effectiveness of
several action items, and how likely the participant was to personally take action on that item. The
scale ranged from -3 (least effective / least likely to take action) to +3 (most effective / most likely
to take action).
Six participants neglected to complete this section, and to maintain the integrity of data collected,
their incomplete answers were not included in the averages for this section.
8/2/2019 In Design Group 3 White Paper
6/19
Figure 3. Total Effectiveness vs. Willingness to Take Action
Participants tended to assign a higher numerical value to their perceived effectiveness of asuggestion than to the likelihood of personally taking action on that suggestion. Both pre-treatment and post-treatment data supported this trend, as noted in Figure 3.
The rated effectiveness of suggestions generally increased post-treatment for participants whoviewed either infographic, with the exception of the fourth listed suggestion: Donating to localfire management organizations. Rated effectiveness scores decreased by 0.2 and willingness totake action decreased by 0.1 post-treatment. (Figures 4 and 5) Willingness to take action inregards to the third suggestion, Taking extra precautions to ensure that campsites are free of trashand debris. also declined 0.3, post-treatment. (Figure 5)
Figure 4: Effectiveness of Suggestions
8/2/2019 In Design Group 3 White Paper
7/19
Figure 5: Likelihood of Taking Action
Participants who viewed either infographic generally rated the effectiveness of suggestions higher
post-treatment. Participants who had viewed the gains-framed infographic reported 0.2 points
higher effectiveness, and those who had viewed the loss-framed infographic responded with 0.1
points higher, post-treatment. (Figure 6)
Figure 6: Effectiveness of Suggestions by Infographic Viewed
Participants who viewed the gains-framed infographic marked their willingness to take action 0.2
points higher than pre-treatment, and participants who viewed the loss-framed infographic ranked
their willingness to take action an average of 0.1 points lower, post-treatment. (Figure 7)
8/2/2019 In Design Group 3 White Paper
8/19
Figure 7: Willingness to Take Action by Infographic Viewed
Survey Analysis
Both infographics were effective to increase the overall effectiveness ratings of suggestions, butpost-treatment, willingness to take action was still significantly lower than rated effectiveness. Thisimplies that these infographics may have increased concern on a public level, but not enough thatparticipants felt a need to take personal responsibility to help prevent wildfires.
Although the rated effectiveness of ensuring that campsites are free of debris increased, the
participants willingness to take action decreased, indicating that the infographics had a negative
effect on participants desire to take action. Because the rated effectiveness and willingness to
donate to local fire management organizations both decreased post-treatment, this indicates that theinfographics were defective in influencing participants to donate money. (Figure 4)
Participants who viewed the gains-framed infographic increased their rated effectiveness ofsuggestions by twice as much as the loss-framed infographic, implying that the gains-framedinfographic was more effective to influence participants to the importance of the suggestions.
The gains-framed infographic increased participants willingness to take action while the loss-framed infographic lowered their willingness to take action. This would indicate that the loss-framed infographic deterred participants from taking action, and could be considered defective inthis aspect. (Figure 3)
Most of the suggestions were rated higher post-treatment in both effectiveness and participant
willingness, indicating that the infographics were minimally effective to increase participants
concern. (Figure 3)
Survey Discussion
8/2/2019 In Design Group 3 White Paper
9/19
Survey results revealed a strong discrepancy between the rated effectiveness of a suggestion andthe willingness of the participant to take action on that suggestion. It might be worth the efforts ofinfographic development to attempt to close this gap, and influence the readers of eitherinfographic to take action. This might be possible if the infographic were developed with morepersonal or region-specific information, such as land management contact information, information
on how to support legislation, or easy methods individuals can use to take preventative measuresagainst wildfires.
Since post-treatment scores declined for participants rated effectiveness and willingness to donatemoney to fire management organizations, it appears that these infographics were defective to givethe participants any sort of fiscal responsibility. It might be concluded that prior to viewing theinfographics, participants may not have known what measures were already in place to managewildfires. After viewing the infographics, participants may have learned that land managementpractices are already in place, and felt less pressure to become personally responsible to offerfinancial assistance. To better influence the target audience, it might be best to modify theinfographics to include information that shows a need for increased financial support and the
benefits of increased funding for wildfire prevention.
Participant Interview Results (See Appendix B)
We collected information about each of the participants in an attempt to draw correlations betweennot only the specific demographics we selected, but others we may not have intended. The figuresbelow are the results of demographic data compared with the survey statement,Im concernedabout wildfires near my community.
All participants pre-treatment rated their wildfire concern between -0.7 and -0.6 points. Post-treatment, participants who viewed the gains-framed infographic increased wildfire concern to apositive score of 0.4. Although participants who viewed the loss-framed infographic increased
their scores by 0.5, their overall wildfire concern still ranked negative. (Figure 8)
Figure 8: Wildfire Concern by Infographic Viewed
8/2/2019 In Design Group 3 White Paper
10/19
Pre-treatment, wildfire concern was an average of 0.0 for Moscow Residents, and negative for allother demographics. Every demographic increased their post-treatment concern levels, but onlyMoscow Residents reported a positive level of concern. (Figure 9)
Figure 9: Wildfire Concern by Demographic
Residents of Idaho and Washington scored negative levels of concern pre-treatment, and positiveconcern post-treatment, while residents of Alaska and Oregon scored neutral and positive concernpre-treatment, but changed to negative concern post-treatment. Residents of California seemedunaffected by the treatment, and rated perfectly negative scores before and after treatment. (Figure10)
Figure 10: Wildfire Concern by State of Residency
Some political responses received may be considered to be errors (i.e. Buddhism is generally notconsidered to be a political view), but all views expressed were considered to best represent theentire group of people tested. Overall, there was a mix between which political affiliations changed
8/2/2019 In Design Group 3 White Paper
11/19
their level of concern. Buddhism, Communist, Democrat and Moderate all reported negative levelsof concern, although Conservative increased concern slightly post-treatment, and Moderatedecreased concern.Democrats and Libertarian decreased concern after treatment, but Independent, Liberal,Republican, and Undecided all increased concern post-treatment. (Figure 11)
Figure 11: Wildfire Concern by Political Views
Participants who ranked their hometown as urban increased their wildfire concern from -1.2 to1.0, but participants who stated that their hometown was rural decreased wildfire concern from-0.3 to -0.4 post-treatment. (Figure 12)
Figure 12: Wildfire Concern by Community Type
Interview Analysis
Those who viewed the gains-framed infographic yielded a positive level of concern, post-
treatment, but participants who viewed the loss-framed infographic, despite an increased level of
concern, still remained negative regarding their concerns about wildfires. This would indicate that
8/2/2019 In Design Group 3 White Paper
12/19
the loss-framed infographic had failed to adequately increase concern about wildfires to a desirable
level. (Figure 8)
Although Moscow Residents rated their wildfire concern higher post-treatment, the other
demographics still failed to express a positive amount of concern for wildfire danger. This may
indicate that either infographic had failed to inspire the majority of our participants to be concerned
about wildfires. (Figure 9)
Data indicating that residents of Idaho and Washington were less concerned before treatment, and
more concerned post-treatment may indicate that they were unaware of wildfire dangers before the
test, but were willing to concede that healthy wildfire concern is necessary post-treatment.
However, residents of Alaska and Oregon, who indicated positive concern pre-treatment, lost
concern, which indicates that the infographics may have been ineffective to generate concern. This
is supported by California residents having completely negative and unaffected concern before and
after treatment. (Figure 10)
Political views and levels of concern seemed to be as randomly increased or decreased, and
although strong correlations could be made between individual groups, groups we would assume
would test at similar levels of concern (i.e. Conservative and Republican) failed to do so. This
would indicate that the size of our test groups was not large enough to be a true sample of the
population. (Figure 11)
As expected, participants from rural communities seemed to have higher levels of wildfire concern
than those of participants from urban communities. However, the infographics seemed to be much
more effective on urban dwellers, since their levels of concern appeared to increase. The fact thatparticipants from rural communities decreased their levels of concern after treatment indicates a
failure for the infographic to raise concern for all demographics. (Figure 12)
Interview Discussion
The infographics could be improved to encourage participants to rank wildfire concern higher
overall. Even though Moscow Residents rated post-treatment wildfire concern much higher than
the other groups, the other demographics failed to yield a positive amount of concern. Despite
increased post-treatment concern, low concern will yield low support of wildfire prevention
measures, indicating that these infographics fail to generate concern.
The survey generating inconsistent scores between demographics from different states would be
worth pursuing as a greater, separate research project. The scores received from our smaller test
indicate either a complete failure of the infographics to generate concern, or a very strong
difference in both knowledge and concern between different states. Further testing would be
necessary, and might be recommended since the intended research will encompass several different
states in the Northwest.
8/2/2019 In Design Group 3 White Paper
13/19
Whether or not there is a true correlation between political affiliation and levels of concern would
also be better determined through a more extensive test. The data we collected on this did not seem
to indicate any true strong correlations since there were very few members tested from any one
political party. However, performing a large-scale test to discover the effectiveness of the
infographics on different political parties might be quite useful if the purpose of the demographics
is to encourage residents of the Northwest to support land management legislation.
Since the infographics failed to increase levels of concern for participants from rural communities,
this could also be an indicator that the infographics information may have educated an already
concerned demographic about fire prevention measures already in place, and helped to ease
concerns about wildfire dangers. This indicates that perhaps the infographics were merely
informative, and did not convey the real and present dangers that individuals living in the
Northwest region could face. The infographics may be improved by ensuring that these dangers are
expressed.
Overall, there is a lack of real evidence to indicate that either of these infographics truly raise
levels of wildfire concern. Most of these tests indicate that at least half of the participants are not
actively concerned about wildfires. The infographics should be modified to concern levels in the
participants. Our study, however, did not perform tests to determine how this could be done.
Other Findings
Participants did not indicate any difficulty or discomfort in answering questions on the interview.
Participants did not indicate any problems with the knowledge quiz.
The test administrators for all groups stated that many participants asked the administrator to
explain what a prescribed burn is. The administrators did not answer the question to ensure
fairness in evaluating knowledge scores, but the question was prevalent enough that it became a
concern.
Participants had several problems with Question #2 on the survey, indicating difficulty in
answering the statements with the given ranking. FarmHouse and Moscow Residents did not write
their concerns on the survey itself, but both the Moscow Residents and FarmHouse test
administrators reported general confusion, and told participants to answer as best they could
regardless.
Argonaut participants written comments included:
This question does not work.
On statement Prescribed fire is too uncontrollable to be a forest management tool,
8/2/2019 In Design Group 3 White Paper
14/19
participant commented, Important in my life doesnt work.
Relabeled categories to disagree, neutral, and agree
On Question #2 instructions, This doesnt make sense.
On statement Prescribed fire is too uncontrollable to be a forest management tool,
participant commented, I dont get it.
Relabeled categories to disagree on left and agree on right.
Scribbled out the categories altogether.
Relabeled categories to disagree, neutral, and agree
Relabeled categories to disagree, neutral, and agree
Commented entire section: I think these questions are better suited to agree or disagree,
and thats how I answered.
Discussion
Effectiveness of Infographics
After the tests were tabulated and analyzed, it was easy to see that the infographic had done its job
effectively; there was a substantial amount of knowledge gained on average by nearly all test
participants. Both infographics resulted in around a 30% gain in knowledge post-treatment
compared to the pre-treatment knowledge test. Since there was an almost equal percentage of
knowledge gain in both the gains and loss frames of the infographic, it cannot be concluded that
one was more effective than the other by a large margin. However, the loss-framed infographic did
not seem to effectively influence participants to increase willingness to take action on any of the
survey suggestions. Survey results showed that the willingness to take action dropped by a small
margin after treatment. Overall, results reveal that the participants seemed to gain more knowledgeand motivation to take action from the gains-framed infographic as opposed to the loss-framed
infographic.
Survey Critiques
As the tests were administered to the participants, it was clear that there was confusion among
them in regards to Melissa Clarks survey: primarily Question #2. Many of the participants voiced
their confusion while taking the survey, and many others wrote notes about how they decided to
answer the questions in a different manner.
After administering the tests, our group reviewed the survey again to analyze how the questions
wording was indeed misleading and confusing.
Question #2 of the survey included statements such as prescribed fire is too uncontrollable to be a
forest management tool, and instead of the options being ranked on an agree to strongly
disagree scale, participants were asked to rate the options on a scale of extremely important to
not important at all. If the scale for this question had been more closely related to the individual
8/2/2019 In Design Group 3 White Paper
15/19
statements, it would have reduced confusion and made the survey a more useful tool to gain
opinions.
In addition to the confusion about this question, some participants overlooked that there wasactually a back side to the survey, and simply did not complete it. For future studies, it might be
beneficial for the test administrator to inform participants prior to taking these tests that the surveyconsists of a double-sided copy.
Limitations of Study
While this test collected a large amount of information, not everything that would be informative
to the development of a large-scale study could be extracted from the data we collected. Even with
our group performing the test on a larger group of participants than was required, the information
we collected only gave results for the information we sought after: to determine if the infographics
were an effective way of conveying knowledge to participants, if the infographics were effective to
change participants views regarding wildfires, and whether a loss-frame or gains-framed
infographic was overall more effective to perform either of these tasks.These tests did not inform us of how the infographics could be improved to increase concern about
wildfires, how to encourage participants to donate money to fire management organizations, how
to help participants feel stronger emotions of loss or gain through the framing of the infographics.
We also did not test a large enough sample of the local population to have only a small margin of
error in our tests.
Recommendations for Melissa Clark
Some aspects of the test require some modification, primarily the survey. Question #2 needs the
evaluation criteria to be modified into reasonable answers to the statements. Test administrators
will need to inform participants about the back page of the survey, or ensure that all surveys given
are printed on one-sided paper.
8/2/2019 In Design Group 3 White Paper
16/19
Appendix A
Gains-based Infographic Loss-based Infographic
8/2/2019 In Design Group 3 White Paper
17/19
Appendix B
Pretest Interview Participant Number .
1. Gender .2. Major .
3. Hometown . State .4. Do you currently live in a rural area, or an urban area? Circle one: Rural/Urban5. Political affiliation/preference .6. Have you had to deal with a wildfire before? Explain:
.
.
.
8/2/2019 In Design Group 3 White Paper
18/19
Appendix C
Knowledge Quiz Participant Number .
1. Approximately, how many wildfires were there in the Northern Rockies (Idaho/Montana)in 2011?
a. 100b. 1000c. 2000d. 10,000
2. What is the projected worth of Idahos forests?a. $13 millionb. $152 millionc. $1 billiond. $3 billion
3. Since the creation of the Forest Service in the early 1900s has wildfire frequency...a. decreased
b. increasedc. remained the samed. is obsolete because wildfire is no longer a problem
4. Since the creation of the Forest Service in the early 1900s has wildfire severity...a. decreasedb. increasedc. remained the samed. is obsolete because wildfire is no longer a problem
5. Prescribed burns, initiated by the Forest Service...a. greatly reduced fire hazardb. reduced wildfire severity
c. consume possible fuel sources (debris, vegetation, etc.)d. all of the above6. What is a measure that homeowners can take to combat wildfire?
a. install fire retardant sidingb. clear roof surface of fire fuelc. keep the yard clear of leavesd. all of the above
8/2/2019 In Design Group 3 White Paper
19/19
Appendix D
(survey added as PDF in Microsoft Word edited version of this white paper)