Occupied! App:!Pilot!Testing!worldofviruses.unl.edu/wp-content/...8April2015.pdf · Occupied!)App:)Pilot)Testing)EvaluationReport) Page)6) notallthestudentsunderstoodtheconnectionbetweentheglow

Occupied! App: Pilot Testing Evaluation Report Page 1

Occupied! App: Pilot Testing UNL iLab App Development

Formative Evaluation

Amy N. Spiegel, PhD April 2015

Biology of Human is funded by the National Center for Research Resources at the National Institutes of Health through the Science Education Partnership Award (SEPA) Grant No. R25OD010506 (2012-‐ 2017). Its content is solely the

responsibility of the authors and does not necessarily represent the official views of NCRR or NIH.


Table of Contents

INTRODUCTION .................................................................................................................................................... 3 PURPOSE ..................................................................................................................................................................................... 3 DESCRIPTION OF PHASES ......................................................................................................................................................... 3

PHASE I .................................................................................................................................................................... 4 PARTICIPANTS ............................................................................................................................................................................ 4 METHODS .................................................................................................................................................................................... 4 LIMITATIONS .............................................................................................................................................................................. 5 RESULTS ...................................................................................................................................................................................... 5 Turning pages ......................................................................................................................................................................... 5 Training hotspot about SEM coloration (p. 2) ......................................................................................................... 5 Tree of life hotspot (p. 7) .................................................................................................................................................... 6 Microbe Close-‐up hotspot (p. 10) ................................................................................................................................... 6 The four microbe “gang” members (p. 13) ................................................................................................................. 6 Candi/ hypha (p. 14) ............................................................................................................................................................ 6 100 trillion (p. 18) ................................................................................................................................................................. 6 Menu itself ................................................................................................................................................................................ 6 Narration toggle .................................................................................................................................................................... 6 ZoomU (mouth, skin, gut) .................................................................................................................................................. 7 Field Guide ................................................................................................................................................................................ 7 About .......................................................................................................................................................................................... 7 Additional notes ..................................................................................................................................................................... 7

SUMMARY ................................................................................................................................................................................... 7 RECOMMENDATIONS ................................................................................................................................................................ 8 REFERENCE ................................................................................................................................................................................ 8

PHASE II .................................................................................................................................................................. 8 PARTICIPANTS ............................................................................................................................................................................ 8 METHODS .................................................................................................................................................................................... 9 LIMITATIONS .............................................................................................................................................................................. 9 FINDINGS AND RECOMMENDATIONS ..................................................................................................................................... 9 I. Typos/Errors ....................................................................................................................................................................... 9 II. Strongly recommended changes in user interface: ........................................................................................... 9 III. Additional suggestions .............................................................................................................................................. 10 IV. Other issues .................................................................................................................................................................... 11

CONCLUSION ............................................................................................................................................................................ 11 DISCUSSION ......................................................................................................................................................... 12 APPENDIX A ......................................................................................................................................................... 13 APPENDIX B ......................................................................................................................................................... 15


Occupied! App: Pilot Testing

UNL iLab App Development: Formative Evaluation

Introduction The Biology of Human project, funded by the National Institutes of Health (NIH) Science Education Partnership Award (SEPA), focuses on helping youth and adults understand themselves by exploring scientific principles that underlie modern research in human biology. Currently in its third year of a five-‐year grant, the project is creating innovative outreach materials for youth, educators and librarians. One of the educational materials that has been developed is a virtual comic book (available on the BioHuman website) targeted to young adolescents, entitled “Occupied!” about two students in biology class who enter the world of microbes and meet the “gang” of bacteria in the human gut that help keep us healthy. A prior formative evaluation study conducted on a draft, print version of the comic indicated that youth reading the comic wanted more information about the microbes, such as where they live in the body, what they actually do, and what they look like (Spiegel, 2014). Consequently, to provide interested readers with more information about the microbes and other topics in the comic, an iPad-‐based app is being developed around the comic. Five senior level undergraduate computer science students enrolled in the UNL Innovation Lab (iLab) Senior Design Capstone Course are working closely with the project PI and other project team members to design this interactive app. Narration to accompany the app was facilitated Adam Wagler and recorded by Ford Clark, both faculty at the UNL College of Journalism and Mass Communications. The app features the voices of students participating in the local middle school afterschool science club sponsored by the BioHuman project.

Purpose The purpose of this evaluation was to trial test the app using a target age group audience to assess usability and understandability in order to make recommendations to the development team about how to improve the user interface.

Description of Phases The trial testing consisted of two distinct phases. In the first phase, with the app about 90% completed, the iLab development team was directly involved in gathering information from the target audience, observing and interacting with youth. The piloting started with a brief group introduction about the app, and then the youth interacted with the app and iPad in a small group. In addition to gathering formal evaluation data about the app, this interaction between the iLab development team and the youth provided the team with first-‐hand feedback about how the youth interacted with the app and with more general information about their target audience. The majority of students involved in the first phase were somewhat familiar with the comic since they had helped review the print version. The feedback from the first phase led to some changes in the app based on the


recommendations resulting from the pilot testing. The second phase, to assess functionality for users who were unfamiliar with both the comic and the app, took place about one month after the first, and involved all new users who interacted with the app individually. The app was fully functional for the second phase. Detailed methods and findings from both phases are provided below.

Phase I This initial pilot testing took place on Feb 25, 2015 during a regularly scheduled afterschool science club, 4:15-‐5:30PM, at a public middle school. The app was approximately 90% complete, with some features still under development, so not all elements were assessed during this pilot testing.

Participants 9 middle school students (6th, 7th & 8th graders; 2 female, 7 male) involved in an afterschool “3D Science” Community Learning Center (CLC) club sponsored by the Biology of Human project participated in this pilot phase. All the students had extensive experience using touchscreen devices.

Methods All data were collected in one data collection period, during a single club meeting (one 75 minute period). Presenters and observers at the meeting included UNL iLab app developers (5 UNL computer science seniors), the CLC facilitators (four BioHuman staff who normally work the afterschool club), the BioHuman project director, the evaluator, and a CLC staff member. The app was briefly presented on overhead projector by the iLab app developers. The narration was turned on, and the presenter paged quickly through the entire comic, stopping briefly to show the first interactive hotspot, but bypassing the rest. Only a small portion of the narration was audible because of the rapidity with which the app was presented. This large-‐group introduction was designed to only briefly introduce the app, thus allowing the youth the opportunity to explore the app on the iPad themselves. Originally, the evaluation plan for this session called for one-‐on-‐one data collection with the available iPads, so that one or possibly two students would work individually with an iPad and a single observer taking notes. However, because of the smaller number of students participating in this particular CLC session, and the availability of three iPads with the app, we decided to create three groups of three students each, with one to two iLab app developers and one to two CLC facilitators/observers assigned to each student group. The youth were instructed to use the app as they wished, and adult facilitators/ observers took notes, answered students’ questions, and when needed, provided help with using the app. This unstructured app exploration time was designed to see how user(s) in the target age group interacted with the app and the different components.


Limitations No formal observational training was provided to the adult observers prior to this session, but they were asked to take detailed notes about student comments and questions, including any difficulties students had with app, what they skipped, incorrect and correct gestures they use, etc (see Appendix A. Field Notes Instrument). The iLab developers were intimately familiar with the app, and had a vested interest in its usability as well. This may have made it more difficult for them to ascertain usability issues as the youth worked with the app. Conversely, their familiarity may have helped them anticipate possible issues to look for as the youth interacted with the different components. Having a group of three students, and two to three observers per student group is not a naturalistic setting within which to use an app. Thus, the student behaviors may be different from what they would do if they explored an app alone, or without someone observing them. Over half of the students involved in this evaluation experience had read the Occupied! comic in paper form, participated in narrating the comic, and/or listened to the narration of the comic as they read an electronic version of the comic and were therefore familiar with the content. Because of this, those students did not explore the comic and app as a new user who had not seen it before would explore it. In addition, some were distracted by the voices in the narration, and they spent time trying to ascertain which of their classmates was speaking during the narration.

Results Overall, the usability of the app was high. The youth were able to easily page through the comic and access the different hot spots. However, there were a few areas that were confusing or problematic. In some cases, the needed modifications are obvious, in others, less so. The feedback is summarized below by each hotspot. More detailed notes by the observers are included in Appendix B and also organized by hotspot.

Turning pages Students enjoyed turning the pages and liked how smoothly this feature worked. Some students spent time playing with this feature and flipping the page rapidly back and forth. Some students who were familiar with the comic paged rapidly through the comic and when doing so, accidently skipped by some pages and missed the interactive hot spots.

Training hotspot about SEM coloration (p. 2) The app developer previewed this first hotspot during the introduction demonstration. When using the app independently, some students pointed at or otherwise indicated that they saw the hotspot, and one asked, “why is it glowing?” Others did not indicate they saw it, and only one activated it without prompting. It may be that students skipped by this hotspot because they thought they had seen whatever there was to see during the demonstration. However, the fact that one student asked about the glowing indicates that


not all the students understood the connection between the glowing as an indicator of a hotspot and the accompanying activity or additional information.

Tree of life hotspot (p. 7) Some students missed this hotspot, but many found it. Of those who tapped on it, one read it all out loud, the others glanced quickly at it and went back to the comic.

Microbe Close-‐up hotspot (p. 10) It appeared that all the students noticed this hotspot and either opened it right away or returned to open it after reading the whole comic. Most appeared to read the text that accompanied the photo image, and one student asked, “What is food gel?”

The four microbe “gang” members (p. 13) All the students appeared to notice these hotspots, and most touched on them to get the descriptive “bios.” In each case, the student tapped every “gang” member, not just one or two. Some students appeared to read the descriptions; others just glanced quickly at the image and text. One student commented that the animation for the hotspot was distracting and should only go once or twice instead of continuously.

Candi/ hypha (p. 14) All the students appeared to notice this hotspot, but several students thought that there were two different hotspots, not just one, and they tried to tap both sides of the animated image. Students spent varying amounts of time looking at the additional information provided through the hotspot. One student remarked, “I am learning so much my mind can’t handle all this.”

100 trillion (p. 18) Not all the students noticed this hotspot, but that may have resulted from skipping this page altogether the first time they were flipping through the comic. This page elicited more comments from students than other pages, with remarks of “Cool!”, “Needs more corn,” and “Wow – that is a lot! A lot of zeros.”

Menu itself Some students readily found the menu; others did not. One student knew to open the menu when s/he wanted to access the narration, but another group did not find the menu when wanting to turn on the sound. For those who tried to open the menu, some used it as intended, but others tried to swipe the button and were thus unable to get it to open at first. One student advised another to “just hit it” to successfully access the menu. When asked about it specifically, some students said they thought it should be bigger but others felt that is was fine as is.

Narration toggle Once students had located the menu, they seemed to understand how to use the toggle appropriately, but not all were able to locate the narration control when they wanted to. Some students needed help with volume control.


ZoomU (mouth, skin, gut) Once they had opened the menu itself, most students were able to readily access this from the menu options. However, a couple students missed this option and had to be directed to use it. Most were able to zoom in on the images as intended, but a few tried to swipe or pinch the image instead of tapping. Once they began the activity, they zoomed all the way into the most magnified image. Some did this very quickly, so did not appear to read much if any of the text. Students appeared to enjoy the images, with comments such as, “Oh my gosh!” “What’s that?” and “Can you zoom in on that?” All students experienced some difficulty returning to the screens that they intended to go to, tapping the “back” button when the “menu” button would have taken them to where they wanted to go next. When asked, students recommended that the buttons be labeled differently, such as “Zoom U menu” to make it more obvious which button would take them back to the three choices

Field Guide This feature was not yet fully functional, so two of the groups did not use it at all. The one group that did click on it, just looked at the menu page with the images but did not explore into the individual descriptions. When asked, the students said they did not recognize that the images were a menu and that they could access additional pages of information by tapping the images.

About The “About” page was not yet functional at the time of piloting.

Additional notes When reviewing the app for this report, I had some trouble with the narration toggle. I turned it off, and then next time I returned to comic, it was on. Then, it indicated that it was on (showed green), but it was off. I could not get the error to happen again, so perhaps it was the result of pressing buttons too quickly. This may be something to investigate further if possible.

Summary This simple, interactive app provides interested readers with additional information and interesting images to accompany the comic. This trial testing indicated that the app seems to be relatively easy to use for the intended audience as well as appealing to them. This was especially true for some of the individuals in this participating group since many had been involved in the narration of the comic and recognized the voices of the characters. Students easily identified most of the access points or “hotspots” as they read through the comic, although students exhibit different styles of using the app. Some students listened to the entire narration through the whole comic first and then went back to access the hotspots. Others flipped rapidly through the comic first, not listening to the narration at all, searching for the hotspots to tap. This trial testing showed that for students with experience with touch screens, even if not with iPads, could readily manipulate the app and access the additional available information.


Recommendations A few changes are recommended to the app as a result of this evaluation.

• Add a “ZoomU Menu” button (or similar) for users to easily return to that menu from the magnified images. In ZoomU, the “back” button and “menu” button were confusing, and students consistently returned all the way to the comic instead of back into ZoomU to continue to explore that feature. So they ended up having to press multiple buttons to get where they wanted and intended to go. A button titled “Zoom U Menu” would be a good way to clarify things for the user.

• Make the menu button easier to find and use. The menu button was not obvious to all users, and was difficult for students to access even if they knew it was there. Some suggestions were to label it “menu” or to make the button larger.

• Consider providing some clue that the images in the Field Guide menu are actually buttons to access additional information. Some students who accessed the Field Guide menu did not recognize that there was additional information to access. Consider adding some animation or directions that say, “tap on microbe for more details” or similar.

• Conduct an additional usability evaluation with users unfamiliar with comic. Because of the limitations present in this study, it is recommended that a second phase usability study with students not familiar with comic be conducted to assess how well the app works with new users.

Reference Spiegel, A. N. (2014). Biology of Human Formative Evaluation: Youth Feedback on Occupied Comic. Lincoln, NE: Center for Instructional Innovation, University of Nebraska-‐Lincoln.

Phase II This second phase was undertaken to identify any issues with usability that were not detected in the first phase of the evaluation. For this phase, changes to the app on the Zoom U feature had been made based on the Phase I findings. In addition, final modifications had been completed and all the app features were fully functional. This pilot testing took place between March 20-‐23, 2015. Because of the time-‐sensitive nature of this information, an earlier version of this section of the report was made available to the development team on March 24. Because the focus of this evaluation is to identify needed changes, I will focus here on recommendations for change rather than include both strengths of the app and recommendations for change.

Participants Six (6) youth, ages 9 – 17 years, with varying levels of experience using an iPad participated in this pilot testing phase. All indicated that they were familiar with iPads (either their own


or that of a family member). Ages and genders were: M9, F11, F11, F14, F14, F17. This was a convenience sample.

Methods Each participant used the app and interacted with the evaluator individually. Each youth was told that the evaluation of the app was being undertaken to assess usability, and were instructed to use and explore the app on his/her own. Each was observed as s/he manipulated the app, and then asked questions about elements s/he had not accessed. At the end, participants were also asked to provide recommendations for elements they found frustrating, confusing, or hard to find. Participation for each youth took between 15 and 30 minutes.

Limitations The app was only available for pilot testing for a limited time on a single iPad. Thus this imposed significant constraints on arranging for, scheduling, and interacting with participants. The participants consisted of a convenience sample of youth volunteers who were available to the evaluator to pilot the app, so they may not be representative of the youth who will be using the app in the future.

Findings and Recommendations These are organized in four sections. The necessary changes, errors in text that are also easy to change, are listed in the first section. The strongly recommended changes, those that would have a significant impact on usability, are listed second. The third section comprises suggested changes that are not critical but were identified by participants as confusing, or as possible areas to improve. The last section includes additional issues to consider, for which changes may or may not be warranted or possible. Within each section, items are listed in order of appearance in comic, with “menu” items last.

I. Typos/Errors • In ZoomU, Skin, X20000, first line should read, “E coli bacteria mostly hang out on

the skin that lines your intestines.” (currently reads “you intestines”). • In ZoomU, Stomach X40000, last line reads, “This trick that has allowed them to

colonize humans for the past 60,000 years.” This is grammatically incorrect, and could be corrected by omitting “that”.

II. Strongly recommended changes in user interface:

Beginning of App • On cover page, put an arrow or other indicator so the user knows to turn the page to

start. Five of the six youth, after opening app, waited for something to happen because

they did not understand that they were supposed to turn the page. They thought the app was still loading. All the other WoV apps start with arrows to indicate that the reader needs to turn the page. This app, aimed at even younger readers, should also have an indication of what to do when you start.


When asked, one participant suggested that having the page curl might work as an indicator that the reader needs to turn the page. Others, when shown the arrow used on one of the other WoV apps, thought that would work well.

Training Hotspot (microbe on page 2) • Make first hotspot more obvious and indicate what the user should do.

Only one of the six youth knew to tap the hotspots to access new information or an activity. When asked, all the rest replied that they thought the glowing/changing colors on the hotspots were just decoration or animation. They did not know that there was something interactive for them to do. The first hotspot, which is also the most subtle, needs to be more obvious (two readers did not find it even after going back to look for a hotspot on that page). It also needs to explicitly encourage the reader to tap it. One user suggested making it move rather than just change color. Another user suggested putting an arrow pointing to it.

Many apps start with some bubbles of instruction to alert the user how to interact and use the app. Adding a bubble that says, “tap here for more information” or “tap glowing images for more information” would let the reader know there is more to the app than just the comic. If having this information in the comic would be too disruptive to the narrative, it could be included at the end of the comic.

Menu • Make the menu more obvious to find and to use.

Only one reader successfully accessed the menu without help. None of the others would have accessed the menu without prompting and/or coaching. A couple noticed the button, but after swiping and getting the iPad menu, did not think to tap instead to access the menu.

Many readers waited until the end to access interactive elements. Perhaps one solution would be to label the menu explicitly and provide information, such as “tap here to access menu,” on the last page of the comic.

III. Additional suggestions

Copyright page • Move copyright page to the end.

Kindle books (ebooks) now mostly have this page at the end of the books so that the reader can get right to the story. This seems like a good precedent to follow. As one youth articulated, having the mostly black and white page of credits be the first thing is an “unrewarding way to start.”

Candi hotspot: • Simplify the ‘Candi’ hotspot on page 14.

Just have the hyphal form of ‘Candi’ be the hotspot, instead of alternating the colorization between the two forms of ‘Candi.” All but one of the readers tapped both images in sequence, thinking that there were two hotspots and two different pages that would open, not just one.


ZoomU: • Label the levels of magnification in ZoomU from the beginning instead of waiting until

all images in a set have been viewed, and have those labels be active (to access corresponding image).

Two participants were interested in going back to an earlier image to compare to another right away but were unable to access it because of the prescriptive nature of this exploration. When asked, four of the participants thought that the magnification labels should be included right away with the first image. Five of the six participants used a swiping motion across the screen to move from the lower magnification images to the next higher level. For most users, this swiping (instead of tapping) was effective, but not for all. When swiping did not work, at least one user then tried unsuccessfully to tap on the purple buttons to see the next image.

Having the purple buttons be active and labeled from the beginning would allow the users more freedom of exploration, and hopefully be less confusing. One user did not understand that the magnification information, when it appeared, corresponded to the earlier images shown. One user said, “have the magnification numbers there right away; otherwise it’s confusing.”

Menu access • Consider adding the menu button to the cover page or not have “THE END” button

return to cover page. Because users return to the cover page when they reach the end of the book, if

they have not found any hotspots or the menu by then, they are then very unlikely to find the menu since it is not available from the cover page.

IV. Other issues • Consider adding a “done” button to the hotspot destinations instead of just

expecting the reader to tap the image to get back to the comic.

• Fix the intermittent bug in the narration toggle. Sometimes the toggle will be green when narration is off and then purple when narration is on (it’s the opposite of what it should be). This error manifest when one of the youth participants was using the app, and once when the evaluator when using the app prior to trial-‐testing.

• Consider switching the x-‐ray images to photographs. One participant found the x-‐

rays confusing. Referring to the x-‐ray mouth image that has teeth colored from white to greens, yellows and oranges, this user commented, “I don’t get the x-‐ray. It has nothing to do with microbes and the different colors make you think you have different microbes on your teeth.”

Conclusion If the goal is for new users to be able to find and access the interactive elements of this app using the menu and hotspots, then the access to these features must be made more obvious. The results of this trial testing suggest that without these changes, it is likely that many users (maybe even the majority) would not access any of the interactive features.


Particularly given the young age of the target audience, making the app as user-‐friendly as possible so that all the information is readily accessed seems critical. The additional recommendations for change may also help make the app easier and less frustrating to use.

Discussion This two-‐phase pilot test provided key information to use in making revisions prior to the initial public release of the Occupied! app. This written report serves as documentation of the evaluation process used and resulting feedback recommended to improve usability. The evaluation activities also included face-‐to-‐face meetings with the iLab development team to further discuss findings, recommendations, and possible modifications.


Appendix A Field Notes Instrument



Appendix B Compilation of comments and observations from Phase I. (Also provided as initial notes to developers on March 3, 2015) Observer Notes

• Group 1 did not listen to narration at all, but paged through quickly to find hotspots. The three youth in this group were all familiar with the comic, so the plot was not new to them.

• For Group 2, first one youth used the app, and then a second youth (who arrived a little later) used the app, so it was more individual exploration. The third youth in this group was familiar with the comic and not interested in exploring the app.

• Group 3 listened to entire narration and then went back to explore hotspots. None of the three students in this group were familiar with the comic prior to this session.

Student comments & questions / Student actions & behaviors 1. Turning pages Group 1

• Worked well for the most part. • Accidently skipped pages quite often. • Liked to play with pages back and forth. • Liked that it was smooth. • “Are you trying to rip it?” [one student to another, who was flipping page rapidly back and forth] • “Is that you?” [to student who narrated story on app]

Group 2 • Skipped quickly first, then played with pages, smiling, “I like the page”

Group 3 • “There is no volume” [iLab app developer then helped turn on sound] • “How long is this thing?” • Easily followed, worked nicely. • Students took turns turning page • Got pages to turn, but students didn’t read. Flipped through fast to get the hang of it, then listened to narration

all the way through.

2. p. 2 (training hotspot) This was previewed in the demonstration, so students may have not explored it because they already knew what happened. Group 1

• Did not touch it, but pointed at it • Noticed right away, didn’t think to touch it • Able to catch it right away

Group 2 • Obvious to first user • Skipped hot spots – did not notice without prompting

Group 3

• Missed first time, might be because shown on TV (reading as well) • Missed first • “Why is it glowing?” [but didn’t tap to try it]

3. p. 7 (Tree of life) Group 1

• Touched hotspot, glanced at enlarged image, went back to comic • Noticed right away, didn’t think to click it • Able to catch it right away


Group 2 • Obvious to first user • Second user missed it the first time, then went back and found it • Went in, but needed to be directed and did not stay in

Group 3 • One student read out loud • Passed it because shown, possibly; also they were reading, went back after reading

4. p. 10 (microbe close-‐up) Group 1

• “What is food gel?” [reading text in hotspot image] • Noticed right away, clicked it and really liked it. • Able to catch and explore

Group 2 • Obvious button to click • Second user clicked it right away • Skipped at first, then went in but did not stay long (~5 seconds)

Group 3 • Skimmed past it (was reading), but went back after reading

5. p.13 (4 “gang” members) Group 1

• Touched each of them to get the descriptions, but quickly clicked back to comic and did not appear to read the “bios.”

• Noticed right away, clicked them all. Enjoyed seeing more info about each person. • Able to catch and explore.

Group 2 • Clicked into and out of all of them • Second user clicked on all the names

Group 3 • “Should only go once – it is distracting; Once or twice” [animation over the word] • “Long name!” • Went back after reading, read all the hot spots.

6. p. 14 (Candi/ hypha)

• “Looks like a piece of candy!” • Clicked on it right away. • Tendency to click the before then after Candi (as if expecting two different hot spots)

Group 2 • Thought the Candi hotspot was 2 hotspots • Clicked in and out, did not spend much time inside • Easily detected hot spot • “How long is this comic?” • “Skipped to the end”

Group 3 • “I am learning so much my mind can’t handle all this.” • One student read hot spot content out loud. • Skipped (reading comic), went back after reading.

7. p. 18 (100 trillion) Group 1

• Skipped by this the first time, since they were flipping quickly through all the pages and not listening to the narration. So they had to be directed to look at it.

Group 2 • Knew to click the animation • “Needs more corn”

Group 3 • “Cool!” [first reaction] • “Wow – that is a lot! A lot of zeros” • Passed by first time (reading), went back through and clicked after reading


Menu itself Group 1

• Did not find menu. • One student was trying to swipe the menu button, and another student advised, “just hit it.” • Suggested it be bigger. • Not too subtle. • Didn’t interact with it until asked to.

Group 2 • Knew open to turn menu for narration • Went through whole comic quickly at first, then went back to beginning and turned on narration

Group 3 • Tried tapping and swiping • Found, used as intended

Narration toggle Group 1

• “Turn off narrating!” • Very self-‐explanatory.

Group 2 • Shrugged at first, then found it • Easy to use • “Why’s it keep doing that?” • Understood how to turn on narration, but struggled to turn it down without help.

Group 3 • “I would read it without narration” • With the audio – “don’t pay as much attention to content” • Did not find it originally. Needed help. • Had to turn on narration

ZoomU (mouth, skin, gut) Group 1

• “Oh my gosh!” • “Oh my gosh, whoa!” • “What is that?” [stomach] • Student hit “back” not “menu” to go back each time. When asked, recommended that the buttons be labeled

differently, such as “Zoom U menu.” • Thought images were “gross, kind of” • “Main” vs “Back” button functions weren’t obvious. • Zooming in by pressing down was obvious. Made sense to choose where to go next. • Seemed intuitive. • Explored all the way. • The menu button (upper left) should say something else than menu

Group 2 • Trying to swipe • Tried to pinch Zoom • “That…that’s nice” • “Whoa!” • “Is it detecting the heat? Is that why it’s that color?” • Use “zoom U home” rather than menu button • “What’s that?” • “Can you zoom in on that?” • “The skin is like diamond shaped” • “What the heck?” • Both students skipped this, and needed to be directed by facilitators. • Once they got into it, they both spent about 5 minutes on it.

Group 3 • “That’s your tongue?” • “Why does it have a sharpe in?” • “That’s a tastebud? Looks like a rotting rose.” • Suggest maybe swipe left and right • Students did not read descriptions


• Clicked on back to get to the other zooms • Found interesting, used as intended • Hit back first, before menu – change “back” to “ZoomU menu”

Field Guide Group 1

• Clicked on it, but just looked at it. When asked, students said they did not recognize that the images were a menu and led to additional pages of information.

• Clicked it in the menu, but thought it was only a picture. • Still buggy, needs to be fixed. • They didn’t think those characters were actual buttons to click

Group 2 • Skipped – both students. • Advised not to use since it is not working

Group 3 • Did not use.

About Group 1

• Doesn’t work very well when they hit the end of the comic. Seemed odd. • Wanted to know why it was blank •

Group 2 • Add color to the person on the About page

Group 3 • Did not use.

Other observations and student comments/ questions

• There was a bug with turning off • “Can you pause and restart it?” • “Pictures should be more realistic” • “Highlight words as narration goes” • “I don’t like comics” • “So you can record anybody?” • “Why have books when you have the internet? Why are most education books so boring and pointless?” • “Is this an iPad?” • “How long is this?” • “So it’s called Biology of Human and these are just other things?” • “How long did it take to make it?” • They were not interested in the interactive aspects of the comic. • Viruses/ bacteria good and bad.

Documents

Occupied! App:!Pilot!Testing!worldofviruses.unl.edu/wp-content/...8April2015.pdf · Occupied!)App:)Pilot)Testing)EvaluationReport) Page)6) notallthestudentsunderstoodtheconnectionbetweentheglow