1

PowerPoint vs. Prezi – The Impact Of Graphic Organizers On Learning From Presentations Jordi Casteleyn, André Mottart & Martin Valcke There is abundant educational research indicating that graphic organizers (knowledge maps, concept maps, or mind maps) have a beneficial impact on learning, but hardly any research has examined this in the context of presentations. This study therefore investigated how graphic organizers – as delivered via presentation software - affect learning outcomes and mediating variables such as self-efficacy, cognitive load and appreciation of the learning material. An experiment was set up during a university course (n=155). E-lectures were designed that shared recorded audio, but differed in the design of the learning material (graphic organizers), considering the presentation software being used. In both research conditions presentation design was in line with the principles of the Cognitive Theory of Multimedia Learning. The results revealed no statistically relevant difference in cognitive load, self-efficacy and knowledge gain, but participants preferred the e-lecture with the graphic organizers to the control version.

multimedia learning; improving classroom teaching; PowerPoint; Prezi; instructional design

Source: Technology, Pedagogy & Education (in press)

2

Introduction

Since Edward Tufte published his essay “The Cognitive Style of PowerPoint” in 2003 (Tufte, 2003), several publications have repeated Tufte’s claim that PowerPoint might turn every presentation into a “boring sales pitch” (Jennings, 2009; Bumiller, 2010; Doumont, 2011). It is therefore relatively strange that no other presentation program has displayed itself as a true substitute for PowerPoint as the software predominantly used for creating presentations. Apple Keynote, OpenOffice.org Impress and other slideware programs may have improved PowerPoint’s user interface and functionality, but they still make use of its paradigm of slides and slideshow. Nor do they divert the approach from “the bulleted lists” to less text-based screens, despite much criticism towards these “bullet points”. In this paper we therefore include all slideware software when referring to PowerPoint.

Alternatively, in April 2009 the web-based presentation tool Prezi was officially launched and immediately it was regarded as a possible ‘PowerPoint killer’ (Genelius, 2011). One of the main differences between Prezi and slideware programs is that Prezi uses a single infinite canvas instead of the traditional set of slides, which allows you to create presentations in a non-linear way. Furthermore, you can group the visual elements into frames and zoom in and out on these different objects (N.N., 2010). At first, Prezi seems to be an innovative product, but the ideas and technology behind it have been around for several years. Flash-based presentations are nothing new to presentation professionals employed by communication and advertising agencies. Moreover, in “Reinventing Comics” the concept of the infinite canvas and the zooming camera were already introduced (McCloud, 2000), and since then many online comics have escaped the boundaries of the conventional page. Furthermore, since 2008, Microsoft Office Labs had been working on ‘pptPlex’, which should give users the power to zoom in and out of slide sections and move directly between slides that are not sequential in a presentation. In April 2011 they finished the project, but apparently it was decided not to include it into the future standard versions of PowerPoint (N.N., 2011). However, Prezi could outperform the alternatives, because it encourages using graphic organizers, which are similar to concept maps and mind mapping. Surprisingly, research into presentations which incorporate this type of visualisations to represent information is scarce.

PowerPoint Presentations: Design Features

Research into PowerPoint has mostly been conducted from the perspective of learning in school settings. Effectiveness of slideware is typically linked with an increase in learning outcome of its viewers: recalling information from slides (Alley, Schreiber, Ramsdell & Muffo, 2006), academic achievement (Susskind, 2007), recall, recognition and transfer of information (Tangen et al., 2011). It is therefore not unexpected that many papers on how to create effective PowerPoint presentations refer to the Cognitive Theory of Multimedia Learning (CTML) (Mayer, 2009). Adopting a cognitive psychological perspective on learning, this theory is based upon three assumptions. The first assumption can be linked back to the dual-coding theory (Paivio, 1991), which suggests that information is processed via two separate channels (auditory and visual). Secondly, each of these channels has a limited capacity, which again can be embedded in a previous theory. It was already indicated that during learning, the working memory experiences intrinsic (associated with the difficulty level of the information), extraneous (related to the manner how the instruction is presented) and germane cognitive load (linked with the efforts needed to process the information into schemas) (Sweller, van Merrienboer & Paas, 1998). The third assumption is that learning is an active process during which learners filter, select, organize and integrate information based upon prior knowledge. Building upon this theoretical framework and confirming its validity with extended research reports, a number of principles for creating effective multimedia learning environments were formulated (Mayer, 2009): the multimedia principle (information is better processed when students are confronted with a

3

text enriched with images than a text only), the temporal contiguity principle (learners process information more effectively and quicker when text and pictorial elements are presented simultaneously instead of successively), the spatial contiguity principle (text should be represented as close as possible to the relevant image), the coherence principle (the effectiveness of learning materials increases when extraneous information such as extra sound is removed), the modality principle (information added to a multimedia representation creates less cognitive load when this information is represented as audio instead of text), the redundancy principle (learners benefit more from animation enriched with audio than from learning material where the audio is additionally represented as text to the learners), and finally, the individual difference principle (the impact of the previous principles is stronger with learners with low-prior knowledge and with high-spatial abilities). It must be noted though that recent research has uncovered some of the limitations of this theory, for instance that text positioning and the degree of animated motion might also have a less positive impact on cognitive load (Austin, 2009). Although additional components might therefore need to be taken into account to solidify the CTML, the relationship between slideware and learning with multimedia has been widely recognized (Kosslyn, 2007; Atkinson, 2008).

Researchers have applied this multimedia learning theory on PowerPoint and focused on

components specific to slideware, but the research findings hardly resulted in general conclusions. Gross and Harmon (2009) examined PowerPoint presentations at a more structural level. However, the data set they draw upon is limited to two case studies. Furthermore, the effects of PowerPoint deck authoring (content cutting, overflow distortion, and slide title flattening) were analysed, but only from a theoretical perspective (Farkas, 2009). Interestingly, the conventional use of a single word or short phrase as slide headline was challenged, and it was argued that a two-line sentence headline identifying the main topic of the slide leads to a statistically significant increase in audience retention (Alley & Neeley, 2005; Alley, Schreiber, Ramsdell & Muffo, 2006). In addition, Garner et al. determine (2009) that this assertion-evidence slide structure (with a sentence headline as assertion and the rest of slide supporting the claim) is more in line with the multimedia learning principles defined by Mayer (2009). Despite the fact that these conclusions have found quite a few supporters, Farkas (2010) asserts that their arguments and empirical evidence used are problematic, and that they are unreasonably biased towards their design ideas. The control slides did only display bad examples of the standard layout of PowerPoint slides, and should have been competently designed standard slides. It would therefore be too strong to maintain that the effectiveness of this new design can exclusively be attributed to the sentence headline, when one of the main differences between the new design and the control slides is the presence of images. These visual representations can be accounted for much of a presentation’s success. Any kind of visual support is effective from a learning point of view, because students who see lectures with slides outperform students who did not see them (Blokzijl & Andeweg, 2007). According to Tangen et al. (2011) however image-rich presentations are more interesting than bullet point presentations. Nevertheless, concerning accuracy, interesting-but-irrelevant images produce worse results than slides with only text. Moreover, in a knowledge domain without an indisputable iconic sign system, caution should be used when images are employed to display more abstract notions, because these images might be misinterpreted (De Westelinck, Valcke, De Craene & Kirschner, 2005). Nonetheless, slides with interesting and relevant images eventually lead to higher accuracy and recall, and therefore the case for the sentence headline in slides still needs to be addressed adequately. Finally, Mackiewicz (2007) examined audience perceptions of fonts in projected PowerPoint text slides, and analysed (2007) the perception of clarity and attractiveness in PowerPoint graph slides. However, due to the low number of participants in these studies (37 students), the assertions that can be found in the study are difficult to generalize.

4

Perceptions of PowerPoint Presentations

There seems to be a considerable amount of agreement among researchers regarding the appreciation of PowerPoint presentations by students. Today’s students’ attitude towards the use of slideware in education is relatively positive (Roehling & Trent-Brown, 2011). They prefer PowerPoint-based classes to traditional lectures; the software is said to have a positive impact on organization and clarity, and seems to be associated with entertainment and interest, with professor likeability, with good professor behaviours (Apperson, Laws & Scepansky, 2008), and motivation to attend classes (Susskind, 2005). There is also no statistically significant difference in attitude towards PowerPoint as a teaching tool in terms of gender (Kahraman, Çevik & Kodan, 2010). However, these results should be regarded in the correct perspective. Objective data indicate that traditional presentations lead to increased auditory retention (Savoy, Proctor & Salvendy, 2009), and that PowerPoint-based lectures ultimately do not affect student behaviour, which suggests a discrepancy between motivational elements and behaviour (Susskind, 2007; Savoy, Proctor & Salvendy, 2009). Moreover, possibly due to the rise of other easily accessible technologies, PowerPoint no longer holds the first position in likeable teaching technologies. It is now being regarded as less enjoyable than video and internet, and there is a prominent negative attitude towards using PowerPoint in terms of enjoyment, but it still gets the highest scores for fostering learning and motivation (Tang & Austin, 2009). In contrast with the students’ slowly declining positive feelings towards PowerPoint, the required quality standards for PowerPoint slides are increasingly set higher. Students expect the slides to be well-constructed (Apperson, Laws & Scepansky, 2008), which is surprisingly reflected in the research finding that PowerPoint presentations that neglect the CTML principles, have a negative impact on learning (Uz, Orhan & Bilgic, 2009). In conclusion, the novelty factor, which had initially spurred the students’ perceptions of PowerPoint’s impact on cognitive learning and classroom interaction, is perhaps slowly dying out (Burke & James, 2008), and now the application of relevant design principles becomes necessary to reach the potential of PowerPoint as a teaching tool. Self-efficacy

A motivational element that has an impact on the effectiveness of a presentation is self-efficacy. From a sociocognitive perspective, individuals are seen as proactive and self-regulating instead of only controlled by biological or environmental forces (Bandura, 1986). In this respect, human behaviour can be better predicted by beliefs in their own competence than the actual skills. Self-efficacy, which holds a central position in these beliefs, can be defined as the individual belief in the possibility to do a specific task or to show specific behaviour (Bandura, 1977). For this reason, self-efficacy can be seen as a major mediator of personal behaviour and behavioural change (Pajares, 2003). In an educational context, self-efficacy is regarded as a principal component of academic motivation and there is abundant research indicating the directly proportional link between a high self-efficacy level and learning performance (Graham & Weiner, 1996). A strong sense of confidence may serve students well when completing an assignment, make them less anxious and give them greater perseverance in the face of adversity. Although ‘beliefs’ have become an important topic in educational research, self-efficacy has hardly been considered in papers discussing the impact of PowerPoint presentations on students. Compared to traditional lectures, PowerPoint-accompanied lectures are easier to follow and understand, making it easier for them to take notes and to study these notes for test preparation. Although PowerPoint thus enhanced students’ self-efficacy, no increase in academic achievement could yet be detected. This pattern can be discerned across several studies (Susskind, 2005; Susskind 2008; Apperson, Laws & Scepansky, 2006).

5

Nonlinguistic Representations and Graphic Organizers

The dual-coding theory argues that knowledge can be stored both in a verbal and imagery form, and the latter form is called ‘nonlinguistic representations’. Creating graphic organizers is an interesting way to help students generate these nonlinguistic representations, because these graphic organizers combine the verbal (using words and phrases) and imagery form (using symbols and arrows). Six graphic organizers have proven to have an educational advantage (Marzano, Pickering & Pollock, 2001). Persons, places, things and events can be represented via descriptive pattern organizers, because these organizers do not follow a specific pattern. Events can also be organized chronologically via time-sequence pattern organizers. Process/cause-effect pattern organizers can be used to represent processes or causal relationships. Specific events with extra information such as setting, particular people, duration, a specific sequence of events and causal relationship can be put into an episode pattern organizer. Generalization/Principle Pattern Organizers include general statements which are supported by relevant examples. The last example of a graphic organizer, concept pattern organizers, is the most general and perhaps for this reason also the one that has received much attention. With concept patterns students attribute characteristics to a word or a phrase (similar to descriptive patterns), and add examples to these characteristics (similar to generalization/principle patterns) in the visual representation. Creating Graphic Organizers

Concept maps (Novak & Gowin, 1984), knowledge maps (O’Donnell, Dansereau & Hall, 2002) and mind maps (Buzan, 2010) have consistently been linked to graphic organizers. Concept maps include concepts usually enclosed in circles or boxes of some type, and relationships between concepts indicated by a connecting line linking two concepts. Words on the line, referred to as linking words or linking phrases, specify the relationship between the two concepts (Novak & Canas, 2008). The way relationships between the individual elements are developed, differentiates concept maps from concept patterns, but the psychological foundations and main idea are identical. Mind mapping also lacks this identification of the relationship between the items. Compared to concept patterns and concept maps, mindmap representations are mostly limited to radial hierarchies or tree structures. Moreover, mind maps are often spontaneously created reflecting what a person knows about a certain topic, whereas concept patterns and concept maps are more carefully constructed. Mind maps also include images, symbols, icons and other imagery elements, but they fulfill a similar function as the one incorporated by the examples in Generalization/Principle Pattern Organizers. As previously mentioned, theory and research promote that graphic organizers are constructed by the learners themselves. In several domains, studying concept maps and knowledge maps has been associated with a significant increase in knowledge retention as compared to straightforward studying text passages, lists and outlines (Nesbit & Adesope, 2006). Learners who are asked to construct collaborative concept maps instead of conventional summaries generate more relations in the task and individual members also perform better in post-test. Especially students with low cognitive abilities benefit from this educational practice. (Haugwitz, Nesbit & Sandmann, 2010). Moreover, concept maps, also influenced the learning outcome and increased positive motivational aspects within the domain of human biology, while it was not affected by potential difficulties due to the usage of digital concept maps (Schaal, 2010). It is also important to note that self-efficacy plays an important role in students’ ability to construct efficient concept maps (Tzeng, 2009). Studying Graphic Organizers

6

In the context of the present study it is however important to focus on empirical results and other related works that research students who learn from preconstructed graphic organizers. For instance, interestingly, when concept maps are applied in the design of e-learning materials, it can promote learning performance and computer self-efficacy. Furthermore, this computer self-efficacy has a positive effect on learning performance (Shaw, 2010). Other research contrasted studying a hierarchically constructed concept map and a concept map with a network-structure (Amadieu, van Gog, Paas, Tricot & Marine, 2009). Learners with a low prior knowledge level equally attain factual knowledge from both types of concept maps, whereas learners with a high prior knowledge level benefit more from a hierarchical concept map. The impact on the acquisition of conceptual knowledge also interacts with prior knowledge levels. High prior knowledge learners gain equal conceptual knowledge from both concept-map structures, whereas low prior knowledge learners preferred the hierarchical concept map. However, for students with a high verbal proficiency level, no differential effect could be observed when studying the impact of preconstructed maps as communication medium (Nesbit & Adesope, 2006).

Although research has indicated the relevance of graphic organizers for classroom instruction, text remains the predominant way to transmit knowledge to students. The latter is a remarkable observation, given the difficulties to develop structure and coherence in knowledge transmitted via this language modality. Remarkable similarities between this research field and the literature on PowerPoint presentations’ design features can be discerned. Both claim that the emphasis on text leads to poorer results, and that a non-text-based approach should therefore be promoted. It is therefore relatively surprising that hardly any researchers have investigated the potential of introducing concept pattern organizers into presentations. A pilot study by Kinchin and Cabot (2007) experimented with concept mapping principles in PowerPoint presentations, and uncovered that students appreciated slides with a bulleted list if the aim was to memorise facts (i.e. a surface approach to learning) and slides based upon concept mapping principles if the aims was to make links between ideas and gain a better understanding of the subject (i.e. a deeper approach to learning). Yet, it was unclear whether graphic organizers in presentations could generate more effective learning. Correspondingly, mind map-based slides used in the teaching of gross anatomy lead to encouraging students to adopt a deeper level of learning. Empirical results showed indeed that the mean scores of females in the mind map group were higher than that for the traditional one (Ghanbari, Javadnia & Abdolahi, 2010). Although these findings are promising, the former study did not mention what was understood by traditional slides, nor did it look into student appreciation of the mind-map-based slides and other factors that have an impact on the efficacy of the tool.

With the launch of Prezi, presentations embracing graphic organizers as a major communication technique, have returned to the limelight of academic research. This also revitalizes the theoretical concept of circumference, which holds a central position in the thinking on communication of Kenneth Burke (1945). He claims that when defining words we always place it in contexts of varying scope. We widen or narrow our scope of analysis, and accordingly, our interpretation of the words changes. Trying to concretise this abstract notion, people have compared circumference to a camera zooming in and out. In essence, Prezi presentations (or any presentations with graphic organizers) employ a similar approach by placing a word (or phrase, or image) in a certain pattern, enlarging or narrowing the scope, and correspondingly creating a nonlinguistic representation of the information presented. Nonetheless, it is unclear whether graphic organizers have an educational benefit when students are not actively producing these graphic organizers and only have to interpret and process them during a presentation.

7

Research Study Research Hypotheses Referring to the theoretical background provided above, we formulate two hypotheses by which we focus on a depending variable (knowledge acquisition) and mediating variables (self-efficacy, appreciation of the visual material, and cognitive load).

• Compared to the non-Graphic Organizers Condition (NGOC), the Graphic Organizers Condition (GOC) will lead to a significantly higher level of knowledge acquisition. Although most research on graphic organizers focuses on students actively creating the visual representations, there are indications that also presenting students visual material with graphic organizers will result in a higher level of learning performance (Schaal, 2010).

• Compared to the non-Graphic Organizers Condition (NGOC), the Graphic Organizers Condition (GOC) will lead to a significantly higher level of self-efficacy, a significantly higher level of appreciation of the visual material, and a significantly lower level of cognitive load. The novelty factor of new teaching technology (Burke & James, 2008) and the increasing negative sentiment among today’s students of using PowerPoint for enjoyment (Tang & Austin, 2009) argue that the GOC will receive a higher level of appreciation. Since the NGOC applies the principles of CTML but nonetheless incorporates textual elements, it is reasonable to suggest that the GOC will cause less cognitive load.

Method PowerPoint is a very complex hybrid of writing, speaking, visual communication, and features of sophisticated software (Farkas, 2009). It is often difficult to discern which elements contribute to a presentation’s success or to determine the relative size of the impact a certain element of a presentation has. To detect the impact of graphic organizers in presentations on learning, we designed two e-lectures (onscreen synchrony of visual material with recorded speech) in which the audio information remained exactly the same but the visual material differed depending on the condition (GCO or NGCO). Context and Participants Our experiment was part of the course ‘Academic Searching, Reading and Writing Skills’, which is part of the compulsory curriculum of Bachelor in Educational Sciences at a Belgian university (2nd year, undergraduate). This course invites students to participate in the discourse of the academic community (writing and evaluating academic papers) by providing them with both a strong theoretical background and various hands-on techniques. Students are expected to take these classes during the 2nd term of the academic year. Our experiment was planned during the last time slot of the term (April 2011). It was announced as an e-lecture which they had to attend in the IT room of the faculty. The session was also embedded into the time schedule of the course. All participants can - therefore – be considered to share the same prior knowledge regarding academic teaching, because the class is situated at the end of 2 years of university training and at the end of the course (which entails that students have the same background information on the content and objectives of this course). The latter is critical to be able to “control” the level of prior knowledge and its potential impact on final learning performance (Amadieu, van Gog, Paas, Tricot & Marine, 2009).

8

Material Design The audio information, which was identical in both conditions, focused on the discussion section of a research paper. The following elements were elaborately discussed: the usual components of a discussion and their rhetorical function, the order of arguments, lexical and grammatical structures typical for a discussion, and the meaning and placing of ‘hedging’. The professor, who also taught the regular classes, recorded the text (2698 words). The audio file was in Dutch, which is the official teaching language and the mother tongue of the majority of the students. The limited number of minutes (21.01 min in total) ensured that there was less chance of “PowerPoint overload” (Yilmazel-Sahin, 2009).

We wanted to study the impact of graphic organizers on learning from a presentation. Since graphic organizers display the structure and relationships between the items of a presentation, the cognitive load in the GOC version should be lower than in the NGOC version, resulting in a positive impact. To guarantee that research conditions only differed at this level, all basic information elements were the same in both conditions, but differed in their design. The NGCO consisted of 47 slides, amounting to 1310 words, and the GCO used 60 different screen displays (it is difficult to label them as “slides” in the Prezi context), using the identical set and number of words as in the NGCO. The NGCO applied the principles of CTML, and the slide titles were phrases, and not two-line sentence headlines (cf. 1.1. for a discussion of this issue). The GCO organized information via Descriptive, Process/Cause-Effect, Generalization/Principle and Concept Pattern Organizers, and to avoid unnecessary additional cognitive load we created the GCO as a hierarchical network. (Amadieu, van Gog, Paas, Tricot & Marine, 2009) We decided not to include images in the conditions, because these might interact with the graphic organizers and as such blur the research design (Tangen et al., 2011). For additional colouring (i.e. lines linking concepts) we used two primary colours (red and blue) on a white background, because these contrasting colours increase readability (Riley & Mackiewiz, 2011). The colour brown was only used 6 times to colour a circle in the GCO, and therefore its impact should be minimal.

To summarize, both conditions used exactly the same ‘presentations items’ (such as words), but were different in their ‘structure’. Illustrative examples of two alternative elaborations of the same content can be found in Figure 1, 2, 3 and 4.

De 5 onderdelen van een Discussion:!van feit naar mening

1.  Communiceren van de belangrijkste resultaten

2.  Vergelijken van de resultaten met vaste criteria

3.  Interpreteren van de resultaten

4.  Mogelijke kritiek pareren

5.  Vermelden van de implicaties

Fig. 1 Illustrative slide of the Non-Graphic Organizers Condition

9

Fig. 2 Illustrative screen shot of the Graphic Organizers Condition

Fig. 3 Slide deck of the Non-Graphic Organizers Condition

10

Fig. 4 Screen shot of the entire canvas of the Graphic Organizers Condition

The e-lectures are also made available online:

• The NGOC condition: http://vimeo.com/30909407 • The GOC condition: http://vimeo.com/30908991

Research Instruments Several research instruments were administered to measure self-efficacy, cognitive load, appreciation of the learning material, and knowledge mastery.

A new scale was designed to determine the self-efficacy related to the topic covered by the presentation, namely the discussion section of research paper. As recommended in the literature, the development of the self-efficacy instrument was grounded in a clear analysis of the competences that reflect final performance (Lachman & Leff, 1989; Pajares, 1996; Pajares & Miller, 1995). Eighteen items were developed that cover mastery of the conceptual knowledge and skills elements. For instance, “I can use different linguistic forms to apply hedging in a research paper”. For each individual item, students were invited to estimate – on a scale of 0 to 100 - the extent they feel sure they master the specific knowledge or skills element. Cronbach’s alfa of this test (α=.898 for the first administration and α=.906 for the second administration) reflect a high reliability.

Cognitive load can be measured on a subjective report of perceived mental effort (Paas, Van Merriënboer & Adam, 1994). Students indicate the amount of effort - when studying the learning material - they experienced on a scale varying from 0 (very easy) to 10 (difficult). This is used in the study as a subjective cognitive load measure. Reliability of this scale is high (Cronbach's α of .90 to .82) (Paas, 1992).

To determine the mastery of knowledge elements related to the content of the presentation (discussion section of a research paper), a knowledge test was developed and reviewed by content specialists. Since the topic discussed in the e-lecture could be linked to content from previous classes, we expected that most students would at least have some of the items correct. The test consisted of 10 multiple choice items. For instance: “What is the objective of a discussion section of a research

11

paper? A. Giving a brief overview of the full study, b. Communicating the way the study was executed, c. Interpreting the findings of the study, or d. Communicating the research hypotheses.” In order to best benefit from the educational potential of graphic organizers (and its display of structure and relationships), different levels of cognition were tested, viz. remembering, understanding, analysing and evaluating. The same test was used in both research conditions.

Several scales can be employed to discover the students’ appreciation rate of multimedia learning material. We decided to adopt scales that build on the Technology Acceptance Model, as developed by Davis (1989 & 1992) and Saade, Nebebe & Tan (2007). The scales focusing on attitude toward behaviour, perceived usefulness, and intrinsic motivation were adopted to develop a clear picture about students’ appreciation of the particular presentation mode. The items of the scales were translated to Dutch, and modified for the specific situation of the research study. The items of the original scales were presented in a random order, and participants were asked to score the different items on a 5-Point Likert scale.

Research Procedure

For this experiment, a classical pretest-posttest design was used, with the GOC version as the experimental sample, and the NGOC version as the control sample. 158 students were enrolled for the course, and were invited to the e-lecture sessions. In this context it is important to mention that these students use computers, Internet and an e-learning environment on a daily basis and are as such familiar with the learning environment set-up.

Due to a strict monitoring of student attendance, there was a high attendance rate (n = 155; age = 19.5 years; 3 males and 152 females). We ensured that the students were randomly and proportionally assigned to a condition, resulting in 78 students taking the condition without the Graphic Organizers and 77 students involved in the condition with the Graphic Organizers. We asked participants to register the amount of time they spent on studying in the e-lecture environment (which was 21.01 min in total). Students from the NGOC (M = 30.22, SD = 3.96) completed the experiment slightly faster as compared to students in the GOC condition (M = 29.61, SD = 2.02).

Students were presented with a booklet with the procedure to follow. When a signal was given students started the session. They first completed the questionnaire focusing on self-efficacy (t1) and a prior knowledge test about the topic dealt with during the e-lecture (t1). After this they participated in one of the e-lectures, which were made available via the familiar online learning environment of the university. After having participated in the e-lecture, students completed a questionnaire focusing on cognitive load, the three scales derived from TAM studies (attitude toward behavior, perceived usefulness, and intrinsic motivation), the self-efficacy scale (t2) and a knowledge test (t2). The knowledge test is a parallel test to the prior knowledge test, presented at the start of the session. Results Descriptive Analysis Table 1 summarizes the descriptive statistics of the pre- (t1) to the post-test (t2) knowledge gain, the pre- (t1) to post-test (t2) self-efficacy changes, cognitive load measures, and appreciation of the e-lecture (attitude towards behaviour, perceived usefulness and intrinsic motivation). Knowledge Gain

The pre-test results suggest that students from both conditions possessed similar prior knowledge levels about the content covered in the e-lecture (t (1, 153) = -0.54, p = 0.594). This guarantees that participants shared a similar background and that groups were comparable. The analysis results also confirm our expectations that some students already master - at the start - a relatively high level of the

12

knowledge since the topic of the e-lecture did build upon concepts discussed in previous classes. When we look at the results of the post-test, a small difference between the two conditions

can be noticed (t (1, 153) = 1.347, p = 0.180). There is a considerable knowledge gain in both groups of participants, but students in the NGOC condition seem to score slightly better. However, the difference between these results appears not to be significant. Mediating Variables

With respect to self-efficacy, no significant differences between the two conditions could be detected. At the pre-test, both groups obtain an almost identical score (t (1, 153) = -0.37, p = 0.709), and at the post-test, similar results were retrieved (t (1, 153) = -0.10, p = 0.922). The same result can be found for cognitive load. Both conditions result in a comparable level of subjective mental effort (t (1, 153) = -0.34, p = 0.737).

Concerning the appreciation of the learning material used in the e-lecture, a significant difference between the conditions can be discerned. GOC scores are larger than NGOC, and this is a consistent pattern found in relation to all measures employed in our study: attitude toward behaviour (t (1, 153) = -3.12, p < 0.01), perceived usefulness (t (1, 153) = -3.21, p < 0.01) and intrinsic motivation (t (1, 153) = -3.00, p < 0.01). INSERT TABLE 1 HERE.

Table 1. Descriptive Results The differential Impact on Knowledge Acquisition

Carrying out an analysis of variance with the two research conditions as factors, post-test knowledge scores as the dependent variable, and the prior knowledge score as a co-variate results in a significant difference (F2 = 7.61, p < 0.01). Students in the Graphical Organizer Condition (GOC) attain significantly lower post-test results. However, the eta square value indicates that this effect is very small (Eta2 = 0.091).

Extending the analysis with the inclusion of the mediating variables as covariates in the analysis, confirms the significant difference in knowledge gains, but again related differences are small (F2 = 2.94, p <0.01, Eta2 = 0.123). We also notice that in the latter analysis, the differences between both conditions are no longer significant (F2 = 2.50, p = 0.12) and outweighed by the significant differences in mediating variables already reported above. Discussion and Conclusions The present study focused on the differential impact of adding graphic organizers to presentations delivered via software. The study additionally focused on a number of additional variables that were expected to mediate in the impact on knowledge acquisition. Considering the research results, a number of critical observations have to be made. First, the results indicate that participants in the two conditions did not report significant different levels in cognitive load. This is partly in conflict with our assumption that builds on the principles of the Cognitive Theory of Multimedia Learning that points at the beneficial impact of multimedia support added to textual information. We therefore expected that students in the GOC condition would experience lower levels of cognitive load. Several reasons can be presented to explain these results. Earlier research stresses the mediating impact of prior knowledge when studying design manipulations of technology-based learning materials (Austin, 2009). However, our pre-test results of knowledge demonstrated that all participants share a similar proficiency in the subject content covered by the e-lecture. It is

13

consequently unlikely that prior knowledge had an impact on cognitive load. Yet, the GOC incorporates animated motion to progress through the graphic organizers. In this context, we can also find an explanation for the unexpected results in the nature of the animations of the graphic organizers. Earlier research suggests that animation increases cognitive load and invokes split attention, resulting in reduced knowledge gains (Austin, 2009). It is plausible that this type of animation has invoked extra extraneous cognitive load. Participants were familiar with the traditional way of structuring lectures with Powerpoint presentations, and were less acquainted with the animated graphic organizers. Therefore they might have been expected to invest considerable additional effort to get accustomed to it. This phenomenon was also discerned when learners were confronted with hypertext structures for the first time (Tremayne, 2008). Moreover, concept maps incorporate an inherent structure that is not always obvious to the learners. These might need explicit guidance to avoid disorientation and resulting extraneous cognitive load (Amadieu, van Gog, Paas, Tricot & Marine, 2009). Furthermore, the present e-lecture was situated within the domain of the social sciences, where an established iconic sign system to develop graphic organizers is not yet available (De Westelinck, Valcke, De Craene & Kirschner, 2005). As a result, the current graphic organizers might have alienated the learners, lowered the impact of the graphic organizers, and increased the extraneous cognitive load to the level of the one experienced in the NGCO. Although we hypothesized that the graphic organizers would lead to an increase in self-efficacy, no statistically significant differences between the two conditions could be discovered. This research finding is rather inconsistent with other studies where comparing PowerPoint-based classes to traditional classes, PowerPoint was said to strengthen students’ self-efficacy (Apperson, Laws & Scepansky, 2006; Susskind, 2005; Susskind, 2008). Several explanations can be put forward to explain the unexpected results of our study. First, the e-lecture might have been too short to expect a considerable impact of the participants’ self-efficacy. Furthermore, the combination of self-efficacy and educational technology is a territory complex to explore. The fact that abundant studies have demonstrated the effectiveness of concept mapping for learning points at a critical difference between the present study and other concept mapping research. In many studies, learners develop these cognitive tools themselves, but this was not the case in the present study. It might be argued that introducing graphic organizers in e-lectures first challenges students’ beliefs about their abilities to successfully accomplish a learning assignment. Once more, due to the relatively short duration of the study it could be that the participants’ self-efficacy was yet not fully restored to experience the full benefits of the graphic organizers.

The impact on the motivational variables results in a clearer picture. Participants appreciated to a higher extent the Graphic Organizers Condition (GOC) as compared to the non-Graphic Organizers Condition (NGOC). This partly confirms hypothesis 2. In general, traditional presentations hardly contain graphic organizers (see “Graphic Organizers in Presentations”). Building on our earlier discussion about the novelty effect of new educational technologies, this first contact with GOC could have invoked a favourable attitude towards this tool (Burke & James, 2008). This preference for GOC can inspire educators and teaching practices. Many instructors have realised the advantages of the self-determination theory, which focuses on the social-contextual conditions that facilitate or obstruct self-motivation of humans. Extrinsic motivation is doing an activity to obtain an external goal, whereas intrinsic motivation refers to initiating an activity for its own because it is interesting and satisfying in itself. The processes corresponding to this intrinsic motivation are interest, enjoyment, and inherent satisfaction. People seem to be more intrinsically motivated for activities that have the appeal of novelty, challenge, or aesthetic value (Ryan & Deci, 2000). Presentations with graphic organizers apparently score better on appreciation surveys than PowerPoint-based presentations, and could therefore appeal to the intrinsic motivation of students. Moving to this new “level” in giving presentations can boost the declining status of PowerPoint in education (Tang, 2010) compared to the

14

high ranking years ago (Bartsch, 2003). It is clear that the novelty effect of these graphic organizers will also gradually wear out, but will possibly be countered by an increased familiarity with these design features, and hence a lowering of cognitive load.

The former elements might have contributed to the present findings that there is no substantial difference in the knowledge gain in the two conditions. Furthermore, research suggests that students with a high verbal proficiency level might experience the beneficial impact of preconstructed concept maps to a lesser extent (Nesbit & Adesope, 2006). This explanation is partly applicable to our study about graphic organizers in presentations, since our participants were all potentially highly verbal proficient. They were all students enrolled in their second year of university, and our experiment took place at the end of the second term, ensuring that all participants were accustomed to attending academic lectures but thus selecting a group of participants that is verbal proficient. This group of students could benefit more from developing this type of graphic organizers themselves. Moreover, research into reading also indicates that graphic organizers have no positive effect on recalling information from advisory reports, which use tree diagrams to indicate the structure. (Lagerwerf, Cornelis, de Geus & Jansen, 2008) A similar finding can be found in our study. The first hypothesis that claimed the GOC would lead to a statistically significant higher level of knowledge acquisition compared to the NGOC is therefore not confirmed.

Further research could build on the earlier remarks, and can also consider additional comments. The individual difference principle of CTML argues that design principles might be valid for particular types of students (i.e. low level of prior knowledge and a high degree of spatial skills). It might be interesting to examine whether this is also of relevance in the field of learning from graphic organizers in presentations. A next study could consider the knowledge domain. The current research was situated within the knowledge domain of social sciences at university level. Other studies might be set up within – for instance – the field of the natural sciences where commonly accepted iconic systems are prevalent and could therefore be more easily integrated in graphic organizers. Another appealing research orientation could be to direct attention to the speaker. Does he/she become more credible when using graphic organizers? Furthermore, how is the self-efficacy of the speaker affected, if (s)he decides to include graphic organizers in the presentation?

It is important to emphasize that in this research we focused on the impact of graphic organizers on learning from presentations. The focus was not to compare PowerPoint (or other slideware programs) to Prezi. Graphic organizers are for instance not unique to Prezi. Moreover, much of the criticism regarding the use of PowerPoint hardly refers to the impact of the visual aids and can mostly be traced back to usability features.

In conclusion, our study results indicate that educators can safely integrate graphic organizers in their presentations. The available design features have no harmful effect on learning, and students seem to prefer presentations with graphic organizers to traditional PowerPoint classes. Most possibly the graphic organizers will even increase the students’ appreciation of the class and eventually their interest in the lecture itself.

15

References

Alley, M. & Neeley, K. A. (2005). Rethinking the design of presentation slides: A case for sentence headlines and visual evidence. Technical Communication, 52(4), 417-426.

Alley, M., Schreiber, M., Ramsdell, K. & Muffo, J. (2006). How the design of headlines in presentation slides affects audience retention. Technical Communication, 53(2), 225-234.

Amadieu, F., van Gog, T., Paas, F., Tricot, A. & Marine, C. (2009). Effects of prior knowledge and concept-map structure on disorientation, cognitive load, and learning. Learning and Instruction, 19(5), 376-386.

Apperson, J.M., Laws, E. L. & Scepansky, J.A. (2008). An assessment of student preferences for PowerPoint presentation structure in undergraduate courses. Computers & Education, 50(1), 148-153.

Atkinson, C. (2008). Beyond bullet points: using Microsoft Office PowerPoint 2007 to create presentations that inform, motivate, and inspire. Redmond, Wa: Microsoft Press.

Austin, K. (2009). Multimedia learning: Cognitive individual differences and display design techniques predict transfer learning with multimedia learning modules. Computers & Education, 53(4), 1339-1354.

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215.

Bandura, A. (1986). Social foundations of thought and action. A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Bartscha, R.A. & M. Cobern, K. (2003). Effectiveness of PowerPoint presentations in lectures. Computers & Education, 41(1), 77-86.

Blokzijl, W. & Andeweg, B. (2007). The effect of text slides compared to visualizations on learning and appreciation in lectures. 2007 IEEE International Professional Communication Conference Proceedings.

Bumiller, E. (2010). We have met the enemy and he is PowerPoint. New York Times, April 26. Burke, K. (1945). A grammar of motives. New York: Prentice Hall. Burke, L. A. & James, K. E. (2008). Powerpoint-based lectures in business education: An empirical

investigation of student-perceived novelty and effectiveness. Business Communication Quarterly, 71(3), 277-296.

Buzan, T. (2010). The mind map book: Unlock your creativity, boost your memory, change your life. Harlow: Pearson Education Group.

Davis, F. D. (1989). "Perceived usefulness, perceived ease of use, and user acceptance of information technology. MISQuarterly, 13(3), 319-339.

Davis, F. D., Bagozzi, R. P. & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22(14), 1111- 1132.

De Westelinck, K., Valcke, M., De Craene, B. & Kirschner, P. (2005). Multimedia learning in social sciences: Limitations of external graphical representations. Computers in Human Behavior, 21(4), 555-573.

Doumont, J-L. (2011). Creating effective presentation slides. OPN Optics & Photonics News, March, 12-14.

Farkas, D.K. (2009). Managing three mediation effects that influence PowerPoint deck authoring. Technical Communication, 56(1), 1-11.

Farkas, D.K. (2010). A brief assessment of Michael Alley’s ideas regarding the design of PowerPoint slides. Retrieved from: http://faculty.washington.edu/farkas/Farkas-AssessmentOfAlley'sSlideDesign.pdf

Garner, J.K., Alley, M., Gaudelli, Allen F. & Zappe, S. E. (2009). Common use of PowerPoint versus the assertion-evidence structure. Technical Communication, 56(4), 331-345.

16

Ghanbari, A., Javadnia F. & Abdolahi, M. (2010). Teaching of gross anatomy for students of medicine by mind map-based power point slides. Medical Teacher, 32(3), 272.

Graham, S. & Weiner, B. (1996). Theories and principles of motivation. In: D.C. Berliner & R.C. Calfee (Eds.), Handbook of educational psychology (pp. 63-84). New-York: Simon & Schuster Macmillan.

Gross, A. G. & Harmon, J. E. (2009). The structure of PowerPoint presentations: The art of grasping things whole. IEEE Transactions on Professional Communication, 52(2), 121-137.

Gunelius, S. (2011). Stand Out From Competitors With Prezi Presentations. Forbes, March 23. Haugwitz, M. Nesbit, J.C. & Sandmann, A. (2010). Cognitive ability and the instructional efficacy of

collaborative concept mapping. Learning and Individual Differences, 20(5), 536–543. Jennings, A. (2009). Creating Marketing Slides for Engineering Presentations. Technical

Communication, 56(1), 14-27. Kahramana, S., Çevika, C. & Kodana, H. (2011). Investigation of university students’ attitude toward

the use of Powerpoint according to some variables. Procedia Computer Science, 3, 1341-1347.

Kinchin, M. & Cabot, L. B. (2007). Using concept mapping principles in PowerPoint. European Journal of Dental Education, 11, 194-199.

Kosslyn, S. M. (2007). Clear and to the point: 8 psychological principles for compelling PowerPoint presentations. Oxford: Oxford University Press.

Lachman, M. E. & Leff, R. (1989). Perceived control and intellectual functioning in the elderly: A 5-year longitudinal study. Development Psychology, 25(5), 722-728.

Lagerwerf, J., Cornelis L., de Geus J. & Jansen, P. (2008) Advance organizers in advisory reports: Selective reading, recall, and perception. Written Communication, 25(1), 53-75.

McCloud, S. (2000). Reinventing Comics. New York: HarperCollins Publishers Inc. Mackiewicz, J. (2007). Perceptions of clarity and attractiveness in PowerPoint graph slides. Technical

Communication, 54(2), 145-156. Mackiewicz, J. (2007). Audience perceptions of fonts in projected PowerPoint text slides. Technical

Communication, 54(3), 295-307. Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works: Research-

based strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development.

Mayer, R. E. (2009). Multimedia learning (2nd ed). New York: Cambridge University Press. Pajares, F. (2003). Self-efficacy beliefs, motivation and achievement in writing: A review of the

literature. Reading & Writing Quarterly, 19(2), 139-158. Nesbit, J. C., & Adesope, O. O. (2006). Learning with concept and knowledge maps: A meta-analysis.

Review of Educational Research, 76(3), 413-448. N.N. (2010). Learn Prezi. Retrieved from: http://ww.prezi.com/learn. N.N. (2011). PptPlex now supports PowerPoint 2010. Retrieved from:

http://www.officelabs.com/Lists/Posts/Post.aspx?ID=95. Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. New York, NY: Cambridge University

Press. Novak, J. D. & Cañas, A.J. (2008). The Theory Underlying Concept Maps and How to Construct

Them. Florida Institute for Human and Machine Cognition: Technical Report IHMC CmapTools. Retrieved from http://cmap.ihmc.us/Publications/ResearchPapers/TheoryUnderlyingConceptMaps.pdf

O'Donnell, A., Dansereau, D., & Hall, R. H. (2002). Knowledge maps as scaffolds for cognitive processing. Educational Psychology Review, 14(1), 71-86.

17

Paas, F., van Merrienboer, J., & Adam, J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills, 79(1), 419-430.

Pajares, F. & Miller, M. D. (1995). Mathematics self-efficacy and mathematics outcomes: The need for specificity in assessment. Journal of Counselling Psychology, 42(2), 190-198.

Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543-578.

Paivio, A. (1991). Dual coding theory: retrospect and current status. Canadian Journal of Psychology, 45(3), 255-287.

Riley, K. & Mackiewicz, J. (2011). Visual composing: Document design for print and digital media. Upper Saddle River, NJ: Prentice-Hall.

Roehling, P. & Trent-Brown, S. (2011) Differential use and benefits of PowerPoint in upper level versus lower level courses. Technology, Pedagogy and Education, 20(1), 113-124.

Ryan, R. M. & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68-78.

Saade, R.G., Nebebe, F. & Tan, W. (2007). Viability of the “technology acceptance model” in multimedia learning environments: A Comparative Study. Interdisciplinary Journal of Knowledge and Learning Objects, 3, 175-183.

Savoy, A., Proctor, R.W. & Salvendy, G. (2009). Information retention from PowerPoint and traditional lectures. Computers & Education, 52(4), 858-867.

Schaal, S. (2010). Cognitive and motivational effects of digital concept maps in pre-service science teacher training. Procedia Social and Behavioral Sciences, 2(2), 640-647.

Shaw, R.-S. (2010). A study of learning performance of e-learning materials design with knowledge maps. Computers & Education, 54(1), 253-264.

Susskind, J.E. (2005). PowerPoint’s power in the classroom: enhancing students’ self-efficacy and attitudes. Computers & Education, 45(2), 203-215.

Susskind, J.E. (2008). Limits of PowerPoint’s Power: Enhancing students’ self-efficacy and attitudes but not their behavior. Computers & Education, 50(4), 1228-1239.

Sweller, J., Van Merriënboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296.

Tang, T.L.-P. & Austin, M.J. (2009). Students’ perceptions of teaching technologies, application of technologies, and academic performance. Computers & Education, 53(4), 1241-1255.

Tangen, J.M., Constable, M. D., Durrant, E., Teeter, C., Beston, B. R. & Kim, J.A. (2011). The role of interest and images in slideware presentations. Computers & Education, 56(3), 865-872.

Tremayne, M. (2008). Manipulating interactivity with thematically hyperlinked news texts: a media learning experiment. New Media Society, 10(5), 703-727.

Tufte, E. (2003) The Cognitive Style of PowerPoint. Cheshire, CT: Graphics Press. Tzeng, J-Y (2009). The impact of general and specific performance and self-efficacy on learning with

computer-based concept mapping. Computers in Human Behavior, 25(4), 989-996. Uz, C., Orhana, F. & Bilgiça, G. (2010). Prospective teachers’ opinions on the value of PowerPoint

presentations in lecturing. Procedia Social and Behavioral Sciences, 2(2), 2051-2059. Yilmazel-Sahin, Y. (2009). A comparison of graduate and undergraduate teacher education students’

perceptions of their instructors’ use of Microsoft PowerPoint. Technology, Pedagogy and Education, 18(3), 361-380.