DEVELOPMENT A RTICLE

Using a semantic diagram to structure a collaborativeproblem solving process in the classroom

Huiying Cai1 • Lin Lin2 • Xiaoqing Gu1

� Association for Educational Communications and Technology 2016

Abstract This study provides an in-depth look into the implementation process of

visualization-based tools for structuring collaborative problem solving (CPS) in the

classroom. A visualization-based learning platform—the semantic diagram for structuring

CPS in a real classroom was designed and implemented. Metafora, the preliminary vehicle

of the semantic diagram, was integrated into the Food and Nutrition CPS curriculum in a

fifth-grade science classroom in east China. Data of a teacher’s and her students’ activities

from the CPS classroom were analyzed to understand how Metafora could be integrated

into the CPS instructional process, what roles Metafora and the teacher played in the CPS

project, and to what extent Metafora might have affected the teacher’s instruction and the

students’ learning activities in the CPS classroom. Results showed that the semantic dia-

gram could be integrated into the CPS classroom adaptively and flexibly, and that it was

important to keep a balance between the role of the semantic diagram and the role of the

teacher. Implications for semantic diagram design and implementation for structuring CPS

in the classroom, as well as future work about the semantic diagram will be discussed.

Keywords Collaborative problem solving � Semantic diagram � Classroom research � The

role of teacher

& Xiaoqing Guxqgu@ses.ecnu.edu.cn; guxqecnu@gmail.com

1 Department of Educational Information Technology, East China Normal University,3663 Zhangshan Road North, Shanghai 200062, China

2 Department of Learning Technologies, College of Information, University of North Texas, UNTDiscovery Park, G150 (G189), 3940 North Elm Street, Denton, TX 76207-7102, USA

123

Education Tech Research DevDOI 10.1007/s11423-016-9445-6

Introduction

Collaborative problem solving (CPS) is an educational approach which has students work

actively together to solve problems (Care and Griffin 2014). Different from the approach of

individual problem solving, CPS echoes Vygotsky’s sociocultural theory in that learners

can gradually internalize collaborative practices as collaborative skills and cognitive

strategies through social interactions (Fischer et al. 2013). CPS has become one of the most

promising educational approaches to help learners acquire knowledge and skills in a digital

age (Hmelo-Silver and Barrows 2008; Stahl et al. 2013). In addition, computer-supported

collaborative learning has caught attention in recent years. Various computer and learning

technologies have been designed to mediate and encourage social activities that promote

both group and individual learning (Stahl et al. 2006).

There have been successful cases of CPS in the classrooms, for instance, the web-based

inquiry science environment (WISE) project (Clark and Linn 2013). However, a lot more

needs to be done to help teachers structure CPS in the classroom. This is because it is not a

simple direct process to translate CPS research to CPS practice. Often, research focuses on

theoretical issues while in practice, teachers must take into consideration various practical

issues including curriculum, discipline, space, energy, and time constraints (Dillenbourg

2013). In addition, CPS is a complex pedagogical approach. The new technologies and new

approaches can all add more work and burden to the teachers, who are already pressured

with time and existing teaching responsibilities (Dimitriadis 2012).Thus, the pedagogical

or technological interventions advocated as having a positive impact on CPS in research

settings may not be readily applicable in real classrooms (Kirschner et al. 2004; Voogt

2008; Roschelle et al. 2013).

Various information and communication technologies (ICT), for instance, web 2.0,

mobile technology, gamification, and wearable technology have been investigated for sup-

porting learning and learners (Johnson et al. 2014). Yet, most studies have focused on

providing specific technological functions for parts of the general learning process, such as

assembling learning resources, sharing information or providing students with specific

experience. In this study, we focus on the design and implementation of visualization-based

tools to structure CPS in real classrooms. There are some theoretical considerations. Visu-

alization tools play an important role in the cognitive and social aspects of learning (Suthers

2001). They can help learners externalize information processing, see patterns, express

abstract ideas with concrete forms, and discover new relationships while solving problems. In

addition, visualization tools can help learners use artifacts to share questions and ideas in

groups, record the groups’ argumentation processes, and communicate conversations of the

group members. In this article, we will report an in-depth case analysis to show how tech-

nological interventions, the visualization tools in particular, can be integrated into a real CPS

science class. The goal is to discover the technological and pedagogical processes needed to

effectively scaffold the visualization tools in a real CPS classroom setting.

Literature review

A collaborative script is a set of instructions designed to structure collaborative learning

(Fischer et al. 2013). Scripting collaborations is a pedagogical method which prescribes the

task and process for a collaborative group (Dillenbourg 1999; Hamalainen et al. 2008).

Collaborative scripts are based on Vygotsky’s concept of the zone of proximal

H. Cai et al.

123

development (ZPD). They are designed to provide supports necessary for the interactive

processes between the collaborators (Kollar et al. 2006). These scripts may include sets of

instructions to help the group members learn how to interact, collaborate, and solve

problems (O’Donnell and Dansereau 1992). Collaborative scripts may vary from macro-

level to micro-level scripts (Dillenbourg and Jermann 2007). The macro-level scripts

concentrate more on setting up conditions to structure collaborative activities and foster the

emergence of knowledge-productive interactions such as argumentation, explanations and

mutual regulation (Dillenbourg and Hong 2008). Examples of macro-level collaborative

scripts include jigsaws scripts (Aronson et al. 1978) and ArgueGraph scripts (Jermann and

Dillenbourg 2003). On the micro-level one focuses more on structuring and regulating the

communication processes in the knowledge-productive collaboration (Villasclaras-Fern-

ndez et al. 2009). Examples of micro-level collaborative scripts are the dialogue script for

facilitating argumentative knowledge construction (Stegmann et al. 2007), or the role-play

script for interaction to promote appropriate questions (Ge and Land 2004; Gu et al.

2015b). Based on the pedagogical notion of collaborative scripts, some researchers have

focused on the design of computer-supported collaborative scripts (Fischer et al. 2007). For

example, the Collage editor allowed generation of hierarchical combinations of collabo-

rative learning flow patterns (Hernandez-Leo et al. 2006). The WISE learning platform

provided different toolkits to pre-design learning sequences to engage students in collab-

orative inquiries (Slotta 2004). Semi-structured communication interfaces were designed to

structure and regulate dialogue to promote communication (Baker and Lund 1997). All

these studies highlighted the importance of considering all aspects—the theory, knowl-

edge, content, pedagogy, and technology in the technology-supported CPS classrooms

(Mishra and Koehler 2006).

In addition, support is necessary to help students engage in complex learning scenarios

such as CPS (Quintana et al. 2004; Kim and Hannafin 2011). Our previous research focused

on the design of the pedagogical intervention framework for CPS in science classrooms (Gu

et al. 2015a). The pedagogical intervention framework helped to improve the students’ social

interaction and problem-solving skills including planning, organizing, and evaluating their

joint tasks. The framework was composed of three parts: making group plans for solving

problems, setting up rules for discourse, and structuring evidenced-based arguments. Making

group plans for solving problems was to guide and enable students to execute a learning task

jointly by planning, organizing, analyzing, and evaluating cognitive development of CPS

(Dawes et al. 2000). Setting up rules for discourse was designed to establish and exercise

group discourse rules (e.g. Dawes 2004; Wegerif and Mansour 2010). The purpose was to

engage students in reflective social discourse and in-depth cognitive processes. Structuring

evidence-based arguments was incorporated to help students identify differing viewpoints,

formulate ideas and beliefs, form judgments and evaluations, select preferred solutions, and

offer evidence and reasons (Cho and Jonassen 2002; Voss et al. 1991).

Prior research has shown promising potential for the technologically-based visual aids

in supporting CPS. There are different types of visualization tools such as those mapping

concepts, organizing graphics, and threading ideas. A concept mapping tool provides

visual cues, such as texts, shapes and line-labeled arrows, to present individuals’ under-

standing of relationships between concepts, or to present individuals’ knowledge structures

(Novak and Canas 2008). Research found that digital concept maps can display shared

knowledge and allow involved learners to focus on information relevant to the problem at

hand, which can help learners solve problems more quickly and accurately (Engelmann

and Hesse 2010). Graphic organizers can be a thinking map or diagram (Hyerle and Alper

2011). Veerman (2001) reported that when students were presented a graphic diagram

Using a semantic diagram to structure a collaborative problem…

123

before discussions, they were able to focus better during the discussions. van Amelsvoort

et al. (2007) also reported that these diagrams functioned as important inputs for the

discussion phase and improved the breadth and depth of the discussions. The visualization

tools such as Idea Thread Mapper helped externalize and trace the processes of CPS. Using

the Idea Thread Mapper, students were reminded to reflect on their own thinking, to be

aware of and incorporate their community’s knowledge, and to make further efforts to

collaborate with their peers (Chen et al. 2013).

Considering the characteristics of visualization-based tools to support CPS, a visual-

ization-based learning environment, named the semantic diagram, was designed to struc-

ture CPS for a real classroom. The semantic diagram can utilize graphics, images, and

other visual elements to visualize the group’s knowledge and understanding of concepts,

principles, and concept relations (Gu and Quan 2014). Following the characteristics of

visualization-based tools mentioned above, the semantic diagram has the potential to: (1)

externalize group students’ understanding of the logic and semantic interrelationships

among different concepts, (2) stimulate the group students’ in-depth discussions based on

the co-constructed learning artifacts during social learning activities and (3) trace the

process of CPS to improve students’ reflection and awareness during collaboration. Our

expectation is that the semantic diagram will offer the needed support for students, while

facilitating their use of the overall CPS learning process because it can record, externalize

and trace both the individual and group learning processes, and can stimulate learners to

think and reflect during the CPS process.

In this study, Metafora (Dragon et al. 2013) was employed as the preliminary vehicle of

the semantic diagram to structure CPS in a real classroom. This is because the design

concept of Metafora is to support collaborative learning within a group (Harrer et al. 2013).

Metafora has two toolkits, the Planning Tool and the LASAD. The Planning Tool can be

used to make visualized problem solving plans. LASAD stands for Learning to Argue:

Generalized Support Across Domains. It is a dynamic discussion-mapping tool for con-

structing and deconstructing arguments. LASAD helps to define the knowledge elements of

Fig. 1 Screenshot of the planning tool in the Metafora platform

H. Cai et al.

123

an argument. The Planning Tool (see Fig. 1) provides a set of icons called Visual Lan-

guage Cards to present different steps for solving problems. These include 12 activity stage

cards and 18 activity process cards. The stage cards include high-level activities such as

exploring a phenomenon while the process cards provide methods and stages including

discussing alternatives. The cards contain titles and visual symbols representing various

learning activities, so that the map created by a group within the Planning Tool can be

considered as a visual language for the students to describe their collaborative work.

Meanwhile, LASAD (see Fig. 2) can provide a learning space for the students to

implement argument rules and to express, discuss, and reflect ideas in a joint learning

environment.

This study focused on how to structure CPS in a real classroom with the semantic

diagram (i.e., Planning Tool and LASAD in Metafora) as the technological support as a

follow-up to the pedagogical intervention framework (Gu et al. 2015a, b). The research

questions are: (Q1) How did the semantic diagram work when it was integrated into the

flow of a real CPS classroom? (Q2) What were the roles of the semantic diagram and the

teacher during the flow of the CPS project? (Q3) How did the CPS project affect the

teacher and the students’ performance?

Research context and methods

Due to the lack of conceptual framework and practical guidelines for structuring CPS with

semantic diagrams in real classrooms, we decided to follow an inductive research

approach, with the intention to derive theories from field data (Barab and Squire 2004). As

such, a case study (Yin 2003) was conducted in an elementary school in east China.

Fig. 2 Screenshot of LASAD in the Metafora platform

Using a semantic diagram to structure a collaborative problem…

123

Participants

The participants were twenty-one-fifth-grade students (8 males, 13 females, mean

age = 9.95 years, SD = 0.59). Since there were five computers equipped with Metafora,

the 21 participants were randomly divided into five groups. There were four groups with

four students, and one group with five students. In addition, the classroom was equipped

with the Internet connection and an electronic projector. The science teacher leading the

class had more than 10 years of teaching experience. After understanding our research

expectations, she co-designed the CPS project supported by Metafora with the research

team which included a lead researcher and three research assistants. She then implemented

the CPS project in her class after completing the design with the research team.

The design of the CPS project

This CPS learning project was co-designed in four rounds of face-to-face communication

(30 min each time) between the science teacher and the research team. Using a co-design

method (Penuel et al. 2007), the classroom teacher not only provided important inputs, she

also had ownership of and familiarity with the instructional design (Lui and Slotta 2014).

In the first round of communication, the teacher was introduced to the research intention

and Metafora. The CPS learning topic Food and Nutrition in the Fifth Grade science

curriculum was identified. In the second round of communication, the teacher was pre-

sented with the initial CPS instructional design by the research team. Four sequential

classroom learning sessions (60 min per session) were identified. They were: (1) to classify

the given food according to nutritional values, (2) to detect the main nutritional compo-

sition of a given food, (3) to discuss the function of a certain nutritional value and (4) to

evaluate a family’s diet for 1 week and develop a healthy diet plan for the family. Several

cycles of revision took place between the teacher and the research team to make sure that

the pedagogical intervention framework (citation omitted for blind review) and Metafora

would be properly integrated into the CPS classroom. Figure 3 below shows the design of

the CPS project.

In the final round of communication, the number of students and the timeline of the CPS

project were decided. The research team was present during the entire process of the CPS

project for design and implementation. During implementation, the research team’s

Fig. 3 The design of Food and Nutrition CPS project

H. Cai et al.

123

primary role was to troubleshoot technical problems such as installing Metafora in the

classroom and helping students to log into the learning platform.

The learning design of the CPS case

Class session 1

Class session 1 contained two parts, using LASAD as the technological intervention. In the

warm-up activity, the pedagogical intervention of setting up rules for discourse used in a

previous study was incorporated (citation omitted for blind review). During the warm-up

activity, the 12 group discourse rules were pre-loaded in the LASAD blocks. Some

examples of the discourse rules were: 1. Listen to others’ ideas; 6. The one who made a

wrong decision should be held responsible; 8. Let the oldest in the group start first. Each

group was directed to categorize the rules into relevant places by moving the LASAD

blocks and make decision on what rule was acceptable and what was not acceptable for

their group. The screen of the learning task in LASAD is shown in Fig. 4.

With the help of LASAD, students were able to identify the collaborative discourses

written in each LASAD block through discussions and negotiations, and they each could

stimulate more discourse by dragging LASAD blocks with the group members.

As shown in Fig. 5, the LASAD interface was designed to structure the learning activity

1 in a collaborative way. At the top of the interface in the ‘‘introduction’’ LASAD block,

there were some pre-designed instructional texts. They were: (1) Analyze the assigned

paper-based nutrition table and write down your idea of food grouping into a specific

LASAD block; (2) Discuss within the group and make your group’s final decision of food

grouping based on every group member’s idea; and (3) Report your group’s decision of

Fig. 4 The interface of the warm-up activity 1. Note 1 Please discuss the following rules within your group.Please categorize and decide which one is good, which is not acceptable, and which you are not sure ofadopting for your group collaboration. Note 2 Rule1: Listen to others’ ideas. Note 3 Rule 6: The one whomade a wrong decision should be held responsible. Note 4 Rule 8: Let the oldest in the group start first

Using a semantic diagram to structure a collaborative problem…

123

food grouping to the whole class. Under the ‘‘introduction’’ LASAD block, there were

several blank LASAD blocks for group members to fill in using their own ideas of food

groupings. This would allow the students to externalize their ideas through visual repre-

sentations based on the learning artifacts so that the group could discuss them together.

This activity set up a foundation for sharing and communication, which enabled the

students to substantiate facts and further investigate each other’s ideas. The members then

made their group’s final decision of food grouping based on the different emergent ideas in

the LASAD blocks. In this session, the semantic diagram could support formative feedback

at the group level.

Class session 2

The goal of class session 2 was to detect the nutrients in food. The Planning tool was

integrated to develop students’ awareness of making plans in solving a problem. In order to

guide students to use the Planning Tool, there was another warm-up activity. The teacher

highlighted the importance of making plans before solving a problem and then introduced

to the students the Visual Language Cards in the Planning Tool. Then in the learning

activity 2, the teacher demonstrated the experiment before the class. Then she allowed each

group to make a plan conducting the experiment using the Planning Tool. Afterwards, each

group conducted the experiment using the physical materials prepared by the teacher.

When the groups finished their experiments, they followed the teacher’s guidance to re-

enter the previous plans on the Planning Tool and make observational reflections or

changes. During this activity, the groups of students not only used the Planning tool to

visualize the process of conducting the science experiment, but also made detailed

reflections on the learning artifacts observed in the Planning Tool.

Fig. 5 The interface of the learning activity 1

H. Cai et al.

123

Class session 3

Learning activity 3, supported by LASAD, was to structure students’ understanding of food

nutrients. In this session, five kinds of printed learning materials about nutrients (protein;

fat; mineral and water; carbohydrates; vitamins) were assigned within each group. The

teacher then guided all groups to type in their understandings of their responsible nutrition

based on the assigned material into the LASAD. Each Group’s interpretations were shared

in the same LASAD interface in real time. In this learning activity, the LASAD application

served as a public knowledge sharing pool at the class-community level. Students could

read what was shown in the LASAD application to gain greater knowledge of food

nutrition, and the teacher could give targeted instructions based on the groups’ visual

interpretations in LASAD. In this session, the semantic diagram supported formative

feedback at the class level.

Class session 4

Learning activity 4 was the main collaborative activity in the entire learning project, which

focused on the intervention structuring evidence-based argument. For structuring this

learning activity, the teacher assigned a hands-on activity for the students to record one-

week of family diet data before learning activity 4 started. With the collected diet data, the

teacher guided each group to discuss and evaluate the family diet in the class. Afterwards,

the teacher asked students to develop a healthy plan using the Planning Tool. Then the

teacher asked two groups to present their plans using the projector in the classroom. During

this time, the teacher encouraged all the students to use the question promote to think and

reflect of their reports. After the whole class discussed their reports, each group returned to

its own group, and made further changes.

Table 1 Excerpt coding examples and results of classroom activities

Time Role Activities Meaning of thepedagogical block

Led by

00:13:50 T Today we will learn about food and nutrition. (Twrites on the blackboard). What questions wouldyou like to ask when you see these words

5. Introduction of thelearning topic

Teacher

S1 Which food does it point to

S2 What is the nutritional value

00:14:50 T Ok. What nutritional value does the food contain?Different kinds of food contain different nutritionalvalues. Today, we will research this question

5. Introduction of thelearning topic

Teacher

00:15:03 T Next, I will assign a table to every group. Thetable shows names and nutritional values of thefood. Please read the table carefully and share whatyou find in your group. After every group finishesthe discussions, I will ask each group to report yourfindings

6. Distribution ofmaterials and grouptasks

Teacher

00:15:10 G (Groups start discussions) 7. Group discussions Student

00:20:02 G (Groups end discussions) 7. Group discussions Student

T teacher, S1 student 1, S2 student 2, G group

Using a semantic diagram to structure a collaborative problem…

123

Data collection and analysis

Data from classroom activity

We used event as a unit of analysis to capture the activities of the teacher and her students

from the CPS classroom. An event refers to an undivided pedagogical episode where a

series of uninterrupted interaction moves with the same semantic content (Prieto et al.

2011; Wen et al. 2015). First, the videos of four classroom sessions were transcribed based

on the words of the teacher and her students (see the first three columns in Table 1). Then

the events were extracted and labeled with the pedagogical meaning of the learning

activities. Afterwards, the person (either the teacher or a student) who led the activity was

noted accordingly. Table 1 shows some excerpts of the coding of the classroom activities.

For example, at the time of ‘‘00:13:50–00:14:50’’, the teacher, student 1, and student 2 had

a conversation. Depending on the content of the conversation, the event could be named as

‘‘introduce the learning topic,’’ which was led by the ‘‘teacher.’’ Because there were four

other events before the time of ‘‘00:13:50’’, the event of ‘‘introduce the learning topic

activity’’ was named number ‘‘5’’. Based on the coding schema, 39 events were identified

in the four classroom video recordings. Two research assistants coded the transcribed data

of four classroom sessions independently. Then their coding results were compared and

negotiated, resulting in 39 events being identified in the CPS classroom.

In order to understand the roles of the semantic diagram and the teacher during the flow

of the CPS classroom, we used the framework of Dillenbourg (2013) to categorize the 39

events. Dillenbourg (2013) claimed that a classroom is a continuum of activities ranging

from intrinsic to extrinsic learning. It included five types of learning activities, namely,

core activities, emergent activities, envelope activities, extraneous events, and the infra

activities from the center to the periphery. Table 2 below shows the category with

examples. For example, the activity, ‘‘2. Discussing group rules on LASAD,’’ was a main

part in the warm-up activity 1; therefore, it was placed in the category of ‘‘Core activity.’’

The event ‘‘6. Logging in the Planning Tool,’’ did not constitute a meaningful part of the

scenario but was necessary to run it, so it was placed into the category of ‘‘Infra activities.’’

Table 2 The coding schema of the pedagogical activity in the CPS classroom continuum

Activitycategories

Descriptions Examples of pedagogicalblock

Coreactivities

Designed as adaptive: the activities of the scenario are pre-defined with certain adaptations to be performed by the systemor by the teacher

2. Discuss group rulesusing LASAD

Emergentactivities

Designed as contingent: some scenarios include activities withcontents unpredictable because they build upon what learnersproduced in earlier phases of the scenario

3. Group report10. Class brainstorm

Envelopeactivity

Routinized: some classroom activities are not part of thepedagogical design but are established school practices

5. Introduce learningtopic

Extraneousevents

Unavoidable: a designed scenario which prepares for unexpectedevents

30. Review the functionof planning tool

Infraactivities

Necessary: activities that do not constitute a meaningful part ofthe scenario but are necessary to run

6. Log in planning tool21. Assign learning

materials

H. Cai et al.

123

The 39 events were further categorized by functions performed to complete the tasks

according to Prieto et al. (2011)’s framework. With this framework, the teacher’s activities

were categorized into ‘‘explanation,’’ ‘‘support,’’ and ‘‘assessment,’’ while the students’

activities were categorized into ‘‘group discussion,’’ and ‘‘group report,’’ and ‘‘others.’’

Different shapes were used to present different categories of activities, as shown in

Table 3.

As shown in Table 3 above, the event ‘‘1. Introduce group rules’’ was performed by the

teacher as a function of explanation, so the coding shape of the event was a circle. The

event ‘‘2. Discuss group rules’’ was performed by the students as a function of discussion,

so the coding shape of that event was a square.

Data from pre-test and post-test on student performance

At the beginning and end of the CPS project, the students were asked to complete the same

tests that evaluated their learning performance. The test contained two parts. The first part

was adopted from Raven’s Progressive Matrices (Raven 1936), which consisted of 15

items. It was used to assess individual’s general cognitive skills (Mercer et al. 1999). The

second part was adopted from the framework of collaborative problem-solving (OECD

2013). A 5-point Likert scale, which contained 16 items, was used to assess student’s

awareness of CPS skills. The internal consistency score (i.e., Cronbach’s alpha) of the

general cognitive skill was 0.72 and the internal consistency score (i.e., Cronbach’s alpha)

of the awareness of CPS skills was 0.86.

Findings

Q1 How did the semantic diagram work when it was integrated into the real CPS

classroom?

Table 3 The coding schema of teacher’s and students’ activity in the CPS project

Roles Activitycategories

Explanations Examples Codingshapes

Teacher’sactivities

Explain Provide an overview ofactivities

1. Introduce group rule29. Review how to make a plan using

the planning tool

Support Provide support forlearning activities

6. Distribute materials. And assigngroup tasks

15. Show experiment tools

Assess Evaluate results oflearning tasks

20. Complete experiments

Students’activities

Discuss Discuss learning tasks 2. Discuss group rules10. Brainstorm

Report Report learning tasks 3. Group report

Other Other student’s activity 13. Observe experiment

Using a semantic diagram to structure a collaborative problem…

123

Figure 6 outlines the pedagogical intervention and technological support in each class

session of the CPS project. It is worth noting that the number in the circle in Fig. 6 refers to

the order of represented learning activities taking place in the classroom. The white circle

refers to the events led by students; while the gray circle refers to the events led by the

teacher. In addition, technology used in a certain step was signaled by a triangle shape with

text description.

It was found that the semantic diagram was adaptively integrated into different learning

tasks in the classroom, under the guidance of the pedagogical intervention framework

(citation omitted for blind review). According to the unique function of each toolkit in

Metafora, the Planning tool and LASAD were selected to load and transfer some parts of

the pedagogical intervention according to the learning context. For example, the Planning

Tool was integrated into class session 2 to accomplish the pedagogical intervention making

group plans (labeled 2). LASAD was integrated into class session 1 to deliver the peda-

gogical intervention setting up rules for collaborative discourse (labeled 1).

In order to optimize the technological support of Metafora for structuring CPS, the

function and use of LASAD and the Planning tool were re-designed and then integrated

into the specific learning task. For example, in the class session 3, instead of acting as the

discussion tool for argumentation, LASAD served as a joint visual learning space for

groups to contribute and share knowledge at the class-level collaboration. In class session

4, instead of acting as the place in the tool for making a plan to solve a problem, this

Planning tool text area served as the workspace for the group to collect and make their

family members’ diet plans visible, which delivered data for the pedagogical intervention

structuring evidenced-based arguments (labeled 3).

Based on the data, we can conclude that Metafora was integrated into the CPS class-

room adaptively and flexibly. In fact, the flexible and adaptive use of the semantic diagram

for different learning scenarios was a critical factor for its successful integration into the

CPS classroom. This suggests that the semantic diagram provided successful visualization-

based toolkits as a group learning platform. It acted as a type of technological scaffolding

Fig. 6 The outline of the pedagogical intervention and technological support in the CPS project

H. Cai et al.

123

or visualization process for different notations of specific pedagogical interventions. It was

also used adaptively and flexibly to organize different learning activities, such as indi-

vidual reflection, group communication and class brainstorming visually.

Q2 What were the roles of the semantic diagram and the teacher during the flow of the

CPS project?

Figure 7 below shows that the Planning Tool and LASAD were mainly used to support

the ‘‘core activity.’’ After an in-depth analysis of each emergent activity, we found that the

emergent activities only took place when the visualized learning artifacts on LASAD or the

Planning Tool were created. Therefore, we conclude that the semantic diagram played a

critical role in supporting and stimulating intrinsic activities during the flow of the CPS

project.

As shown in Fig. 7, most of the gray circles, which referred to the events led by the

teacher, were located in the last three rows. During the class observations, we also noticed

the indispensable role of the teacher. The teacher led the classroom progress, assigned

learning tasks to the students, and coordinated the students’ activities at the class level.

Therefore, we conclude that the teacher played a crucial role in the extrinsic learning

activities during the flow of CPS project.

Last but not the least, further analysis indicated the importance of a balanced role

between the semantic diagram and the teacher. As shown in Fig. 7, most of emergent

activities took place based on the externalized artifacts of the semantic diagram. Towards

the end of those emergent activities, the teacher sometimes provided guidance to lead sub-

activities, such as the flow of 8–9–10, the flow of 23–24–25. Sometimes the teacher made

decisions to stop the activities, such as the flow of 3–4, the flow of 26–27, and the flow of

38–39. Therefore, although the semantic diagram played a critical role in the core activ-

ities, the teacher was indispensable in the classroom in providing the focus for the class.

These findings were confirmed by the class observations. When the groups collaborated in

Metafora, the teacher looked around in the classroom and gave specific guidance to certain

groups. During the process, she paid attention to the groups’ learning status, which helped

her facilitate and guide the sub-activities. In conclusion, the results suggest that, in order to

make the flow of a CPS project move successfully and seamlessly, it is important to keep a

good balance between the role of the semantic diagram and the role of the teacher.

Q3 How did the CPS project affect the teacher and the students’ performance?

Figure 8 below shows that the teacher had a higher percentage of non-lecture activities.

For instance, the teacher had more question promoting activities (5/16, 43.75 %), and

Fig. 7 The pedagogical flow of the CPS project

Using a semantic diagram to structure a collaborative problem…

123

evaluative activities (5/16, 37.5 %), than lecture-oriented activities (6/16, 18.75 % for

introducing activities).

This reveals that the teacher acted as learning facilitator rather than knowledge trans-

mitter or classroom lecturer in the CPS classroom. This finding was consistent with the

classroom observations. When students were busy with the learning activities in Metafora,

the teacher was freed from the knowledge transmission. The teacher had more time to

watch and guide the groups’ discussions and reports, and to provide detailed instructions

for activities. As also shown in Fig. 8, there were 23 students’ activities and more than 16

teacher’s activities. The students’ activities including the group reports and group dis-

cussions, comprised of 6/39 (15.4 %) and 12/39 (30.8 %) of all the classroom activities

respectively. There were more group-centered learning activities than the other kinds of

activities. This means that in this CPS project, students had many more opportunities for

communication and collaboration. Compared to the traditional teacher-centered lecture

approach, this project showed a trend towards student-centered learning.

In order to investigate the effect of the CPS project on the students’ performance, a

paired-sample T test of the students’ pre-tests and post-tests scores was employed. Mean

scores and standard deviations of individual students’ general cognitive skills and learning

awareness of the CPS skills are presented in Table 4.

As shown in Table 4, no significant difference existed between pre-test and post-test in

terms of the general cognitive skills (t = 1.36, P = 0.190[ 0.05). This indicates that the

intervention of CPS project did not influence the development of students’ general cog-

nitive skills. This result is expected because students’ cognitive skills could not be

improved in such a short intervention time period. However, there existed significant

difference of students’ learning awareness of CPS skills (t = 2.81, P = 0.012\ 0.05).

This means that after the intervention of the CPS project, students’ understanding of how to

solve problems collaboratively was greatly improved. This result confirms that the

designed CPS project was indeed student-centered, and that it gave the students a greater

opportunity to experience CPS in the classroom.

Fig. 8 The flow of the teacher’s and students’ activities in the CPS project

Table 4 The paired-sample T test of students’ pre-test and post-test in the CPS project

Pre-test Post-test t Df P

Mean SD Mean SD

General cognitive skills 10.11 2.42 10.53 2.39 1.36 18 0.190

Awareness of CPS skills 58.74 8.57 66.42 13.85 2.81 18 0.012

H. Cai et al.

123

Discussion and conclusion

In this study, we analyzed and reported the process of integrating the semantic diagram

(i.e., Metafora) into a real CPS classroom. The study revealed the relationships between the

semantic diagram (e.g. Metafora), the teacher and the students in the CPS classroom. In the

following, we will discuss the insights and limitations of the study, and we will propose

future work for the semantic diagram.

This study helped us better understand the characteristics of the semantic diagram for

structuring CPS in a real classroom. First, the semantic diagram should be treated as a

toolkit. As such, the toolkit can be adapted into different learning scenarios to structure the

CPS based on the requirements of learning tasks. This design process resonates with the

idea of authoring tools provided in the WISE learning platform (Slotta 2004). Second, the

semantic diagram interface can be redesigned to deliver pedagogical interventions using

collaborative scripts. This design approach resonates with the representational scripting

(Slof et al. 2013). Third, the technology function of the semantic diagram can structure the

CPS activities across the individual, group, class levels. Take, for example, the function of

LASAD in the CPS project. In the warm-up activity 1, LASAD was employed as a digital

whiteboard to show the predesigned learning materials for the students. In learning activity

1, LASAD served as a group opinion pool to record ideas from each member in the group.

In learning activity 3, LASAD served as a joint learning space for each group to contribute

and share knowledge at the class-level collaboration. The reflection of technological

considerations may provide promising answers that will address the pedagogical require-

ments coming from curriculum, the teacher and classroom culture. For students, it can also

record, externalize and trace their learning progresses for further instructions.

In addition, this study helped us better understand the role of the teacher in the CPS

classroom project. First, although the semantic diagram played a key role in the CPS core

activities, the role of a teacher should not be overlooked. These findings confirm Suthers

(2005)’s claim that ‘‘the technological affordance should attempt to leverage the unique

opportunities provided by technology rather than replicating support for learning that could

be done by teacher or (worse) trying to force the technology to be something for which it is

not well suited’’ (p. 666). Second, the teacher played a vital role in the design phrase of the

CPS classroom project because she was familiar with the domain knowledge and familiar

with her students’ learning styles and progress. The co-design with the teacher helped the

researchers to target specific domain knowledge, identify the requirements of technological

support, and streamline the sequence of designed activities into the CPS classroom. This

process is in accordance with statements made by the other scholars for structuring CPS in

a real classroom. Dillenbourg and Jermann (2010) and Kollar et al. (2011) stated that (1)

the curriculum sets the starting points for activities; (2) the learning environment supports

the collaboration; (3) the teacher pre-designs the structure of the learning processes that are

orchestrated in real time, and (4) the learners should be given sufficient freedom to co-

construct knowledge. This project proves that it is important to include the teacher in the

design process for the technologies to be truly beneficial for the teacher and the students.

This study has several limitations. First, the study was a case study of only one

classroom, including one science teacher with 21 students. Future studies should examine

other classrooms to see if similar results may emerge. Second, in this study, we set out to

ask students to take turns entering their ideas into Metafora, and then discuss the accu-

mulated ideas. When one student was typing, the other students were sitting and watching.

We observed that some students gave suggestions or asked questions to the person who

Using a semantic diagram to structure a collaborative problem…

123

was typing. Yet, we did not record such offline activities in this study, which could be

interesting for future studies. Third, we looked into the students’ learning achievements at

the macro level, including their general cognitive skills and their CPS skills. We did not

examine their micro-level abilities, for instance, the students’ cognitive development of the

domain knowledge. Future studies should investigate how to support students’ cognitive

development in the CPS classroom structured by the semantic diagram. Last but not the

least, we did not examine students’ willingness of using Metafora, a factor that could have

affected the results. This study, however, is an important step for us to understand the role

of the semantic diagram, to understand the effective integration of this visualization-based

platform into a CPS project, and to understand how to structure technologies into the CPS

project to create a student-centered learning environment.

Compliance with ethical standards

Funding This study is supported by Chinese National Social Science Foundation (Grant Number: BCA120024), and partly supported by Program for New Century Excellent Talents in University (Grant Number:NCET-11-0140).

Conflict of Interest The authors declare that they have no conflict of interest.

References

Aronson, E., Blaney, N., Sikes, J., Stephan, G., & Snapp, M. (Eds.). (1978). The jigsaw classroom. BeverlyHills: Sage.

Baker, M., & Lund, K. (1997). Promoting reflective interactions in a CSCL environment. Journal ofComputer Assisted Learning, 13(3), 175–193.

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of theLearning Sciences, 13(1), 1–14.

Care, E., & Griffin, P. (2014). An approach to assessment of collaborative problem solving. Research &Practice in Technology Enhanced Learning, 9(3), 367–388.

Chen, M. H., Zhang, J., & Lee, J. (2013). Making collective progress visible for sustained knowledgebuilding. In N. Rummel, M. Kapur, M. Nathan, & S. Puntambekar (Eds.), Proceedings of the Inter-national Conference of Computer-Supported Collaborative Learning: To See the World and a Grain ofSand: Learning Across Levels of Space, Time, and Scale (pp. 81–88). Madison: International Society ofthe Learning Science.

Cho, K. L., & Jonassen, D. H. (2002). The effects of argumentation scaffolds on argumentation and problemsolving. Educational Technology Research and Development, 50(3), 5–22.

Clark, D. B., & Linn, M. C. (2013). The knowledge integration perspective: Connections across research andeducation. In S. Vosniadou (Ed.), Handbook of Research on Conceptual Change (2nd ed.,pp. 520–538). New York: Routledge.

Dawes, L. (2004). Talk and learning in classroom science. International Journal of Science Education,26(6), 677–695.

Dawes, L., Mercer, N., & Wegerif, R. (Eds.). (2000). Thinking together: A programme of activities fordeveloping speaking, listening and thinking skills for children aged 8–11. Birmingham: ImaginativeMinds Ltd.

Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collabo-rative-learning: cognitive and computational approaches (pp. 1–19). Oxford: Elsevier.

Dillenbourg, P. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492.Dillenbourg, P., & Hong, F. (2008). The mechanics of CSCL macro scripts. International Journal of

Computer-Supported Collaborative Learning, 3(1), 5–23.Dillenbourg, P., & Jermann, P. (2007). Designing integrative scripts. In F. Fischer, I. Kollar, H. Mandl, & J.

M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 275–301). New York:Springer.

Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. In M. S. Khine & I.M. Saleh (Eds.), New science of learning (pp. 525–552). New York: Springer.

H. Cai et al.

123

Dimitriadis, Y. A. (2012). Supporting teachers in orchestrating CSCL classrooms. In A. Jimoyiannis (Ed.),Research on e-Learning and ICT in education (pp. 71–82). New York: Springer.

Dragon, T., Mavrikis, M., McLaren, B., Harrer, A., Kynigos, C., Wegerif, R., & Yang, Y. (2013). Metafora:A web-based platform for learning to learn together in science and mathematics. IEEE Transactions onLearning Technologies, 6(3), 197–207.

Engelmann, T., & Hesse, F. W. (2010). How digital concept maps about the collaborators’ knowledge andinformation influence computer-supported collaborative problem solving. International Journal ofComputer-Supported Collaborative Learning, 5(3), 299–319.

Fischer, F., Kollar, I., Mandl, H., & Haake, H. M. (Eds.). (2007). Scripting computer-supported collabo-rative learning: Cognitive, computational and educational perspectives. New York: Springer.

Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56–66.

Ge, X., & Land, S. M. (2004). A conceptual framework for scaffolding ill-structured problem-solvingprocesses using question prompts and peer interactions. Educational Technology Research andDevelopment, 52, 5–22.

Gu, X., Chen, S., Zhu, W., & Lin, L. (2015a). An intervention framework designed to develop the col-laborative problem-solving skills of primary school students. Educational Technology Research &Development, 63(1), 143–159.

Gu, X., & Quan, G. (2014). The literature review of visual knowledge representation and model by semanticdiagram. e-Education Research, 5, 45–52. (In Chinese).

Gu, X., Shao, Y., Guo, X., & Lim, C. (2015b). Designing a role structure to engage students in computer-supported collaborative learning. The Internet and Higher Education, 24, 13–20.

Hamalainen, R., Oksanen, K., & Hakkinen, P. (2008). Designing and analyzing collaboration in a scriptedgame for vocational education. Computers in Human Behavior, 24(6), 2496–2506.

Harrer, A., Pfahler, K., De Groot, R., & Abdu, R. (2013). Research on collaborative planning and reflection-methods and tools in the Metafora project. In D. Hernandez-Leo, T. Ley, R. Klamma, & A. Harrer(Eds.), Scaling up learning for sustained impact (pp. 139–150). Berlin Heidelberg: Springer.

Hernandez-Leo, D., Villasclaras-Fernandez, E. D., Asensio-Perez, J. I., Dimitriadis, Y., Jorrın-Abellan, I.M., Ruiz-Requies, I., & Rubia-Avi, B. (2006). Collage: A collaborative learning design editor based onpatterns. Journal of Educational Technology & Society, 9(1), 58–71.

Hmelo-Silver, C., & Barrows, H. (2008). Facilitating collaborative knowledge building. Cognition andInstruction, 26(1), 48–94.

Hyerle, D., & Alper, L. (Eds.). (2011). Student successes with thinking maps. Thousand Oaks: Corwin Press.Jermann, P., & Dillenbourg, P. (2003). Elaborating new arguments through a CSCL script. In J. Andriessen,

M. Baker, & D. Suthers (Eds.), Arguing to learn (pp. 205–226). Netherlands: Springer.Johnson, L., Adams, B. S., Estrada, V., & Freeman, A. (2014). NMC horizon report: 2014K-12 Edition.

Austin, TX: The New Media Consortium.Kim, M. C., & Hannafin, M. J. (2011). Scaffolding problem solving in technology-enhanced learning

environments (TELEs): Bridging research and theory with practice. Computers & Education, 56(2),403–417.

Kirschner, P., Strijbos, J. W., Kreijns, K., & Beers, P. J. (2004). Designing electronic collaborative learningenvironments. Educational Technology Research and Development, 52(3), 47–66.

Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts: A conceptual analysis. EducationalPsychology Review, 18(2), 159–185.

Kollar, I., Hamalainen, R., Evans, M., DeWever, B., & Perrotta, C. (2011). Orchestrating CSCL-more than ametaphor. In H. Spada, G. Stahl, N. Miyake, & N. Law (Eds.), Proceedings of the Conference onComputer Support for Collaborative Learning: Connecting Computer-Supported CollaborativeLearning to Policy and Practice (pp. 946–947). Madison: International Society of the LearningScience.

Lui, M., & Slotta, J. D. (2014). Immersive simulations for smart classrooms: Exploring evolutionaryconcepts in secondary science. Technology, Pedagogy and Education, 23(1), 57–80.

Mercer, N., Wegerif, R., & Dawes, L. (1999). Children’s talk and the development of reasoning in theclassroom. British Educational Research Journal, 25(1), 95–111.

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework forteacher knowledge. Teachers College Record, 108(6), 1017–1054.

Novak, J. D., & Canas, A. J. (2008). The theory underlying concept maps and how to construct and usethem. Technical report. Pensacola: Institute for Human and Machine Cognition.

O’Donnell, A. M., & Dansereau, D. F. (1992). Scripted cooperation in student dyads: A method foranalyzing and enhancing academic learning and performance. In R. Hertz-Lazarowitz & N. Miller

Using a semantic diagram to structure a collaborative problem…

123

(Eds.), Interaction in cooperative groups: The theoretical anatomy of group learning (pp. 120–141).London: Cambridge University Press.

OECD. (2013). PISA 2015 collaborative problem solving framework. OECD Publishing. http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20Problem%20Solving%20Framework%20.pdf.

Penuel, W. R., Roschelle, J., & Shechtman, N. (2007). Designing formative assessment software withteachers: An analysis of the co-design process. Research and Practice in Technology EnhancedLearning, 2(1), 51–74.

Prieto, L. P., Villagra-Sobrino, S., Jorrın-Abellan, I. M., Martınez-Mones, A., & Dimitriadis, Y. (2011).Recurrent routines: Analyzing and supporting orchestration in technology-enhanced primary class-rooms. Computers & Education, 57(1), 1214–1227.

Quintana, C., Reiser, B. J., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., & Soloway, E. (2004). Ascaffolding design framework for software to support science inquiry. The Journal of the LearningSciences, 13(3), 337–386.

Raven, J. C. (1936). Mental tests used in genetic studies: The performances of related individuals in testsmainly educative and mainly reproductive. Unpublished Master’s Thesis, University of London.

Roschelle, J., Dimitriadis, Y., & Hoppe, U. (2013). Classroom orchestration: Synthesis. Computers &Education, 69, 523–526.

Slof, B., Erkens, G., Kirschner, P. A., & Helms-Lorenz, M. (2013). The effects of inspecting and con-structing part-task-specific visualizations on team and individual learning. Computers & Education,60(1), 221–233.

Slotta, J. D. (2004). The web-based inquiry science environment (WISE): Scaffolding knowledge integra-tion in the science classroom. In M. C. Linn, E. A. Davis, & P. Bell (Eds.), Internet environments forscience education (pp. 203–231). NJ: Routledge.

Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historicalperspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409–426).Cambridge, UK: Cambridge University Press.

Stahl, G., Law, N., & Hesse, F. (2013). Collaborative learning at CSCL 2013. International Journal ofComputer-Supported Collaborative Learning, 3(8), 267–269.

Stegmann, K., Weinberger, A., & Fischer, F. (2007). Facilitating argumentative knowledge constructionwith computer-supported collaboration scripts. International Journal of Computer-Supported Col-laborative Learning, 2(4), 421–447.

Suthers, D. D. (2001). Towards a systematic study of representational guidance for collaborative learningdiscourse. Journal of Universal Computer Science, 7(3), 254–277.

Suthers, D. D. (2005). Technology affordances for intersubjective learning: A thematic agenda for CSCL. InT. Koschmann, T. Chan, & D. D. Suthers (Eds.), Proceedings of the Conference on Computer Supportfor Collaborative Learning: Learning 2005—The Next 10 years! (pp. 662–671). UK: Routledge.

van Amelsvoort, M., Andriessen, J., & Kanselaar, G. (2007). Representational tools in computer-supportedcollaborative argumentation-based learning: How dyads work with constructed and inspected argu-mentative diagrams. The Journal of the Learning Sciences, 16(4), 485–521.

Veerman, A. L. (2001). Computer-supported collaborative learning through argumentation. Doctoral Dis-sertation. Utrecht University, Utrecht. http://dspace.library.uu.nl/bitstream/handle/1874/798/full.pdf?sequence=1.

Villasclaras-Fernndez, E. D., Isotani, S., Hayashi, Y., & Mizoguchi, R. (2009). Looking into collaborativelearning: Design from macro-and micro-script perspectives. In V. Dimitrova, R. Mizoguchi, B. duBoulay, & A. Graesser (Eds.), Artificial intelligence in education: building learning systems that care:from knowledge representation to affective modelling (pp. 231–238). Amsterdam: IOS Press.

Voogt, J. (2008). IT and curriculum processes: Dilemmas and challenges. In J. Voogt & G. Knezek (Eds.),International handbook of information technology in primary and secondary education (pp. 117–132).New York, NY: Springer.

Voss, J. F., Wolfe, C. R., Lawrence, J. A., & Engle, R. A. (1991). From representation to decision: An analysisof problem solving in international relations. In R. J. Sternberg & P. A. Frensch (Eds.), Complex problemsolving: Principles and mechanisms (pp. 119–158). England: Lawrence Erlbaum Associates.

Wegerif, R., & Mansour, N. (2010). A dialogic approach to technology-enhanced education for the globalknowledge society. In M. S. Khine & I. M. Saleh (Eds.), New science of learning: Cognition, com-puters and collaboration in education (pp. 325–340). New York: Springer.

Wen, Y., Looi, C. K., & Chen, W. (2015). Appropriation of a representational tool in a second-languageclassroom. International Journal of Computer-Supported Collaborative Learning, 10(1), 77–108.

Yin, R. (Ed.). (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks: Sage.

H. Cai et al.

123

Huiying Cai is a Ph.D. Candidate and Research Assistant working in the Educational Technology Lab, EastChina Normal University.

Lin Lin is an Associate Professor of Learning Technologies at University of North Texas. Her researchinterest lies at the intersection of new media and technologies, cognitive psychology, and education.

Xiaoqing Gu is now a Professor of Educational Technology in School of Educational Science, East ChinaNormal University, China. Her main research interests are learning science, learning design and CSCL.

Using a semantic diagram to structure a collaborative problem…

123