The 2 Designing Computational Models of Collaborative Learning … · 2017-10-27 · Designing Computational Models of Collaborative Learning Interaction: Introduction to the Workshop

. . . . .

.. . . . ITS 2004 Workshop W9

The 2nd International Workshop on Designing Computational Models of Collaborative Learning Interaction

August 31, 2004

Maceió, Brazil

. . . . . .

.. . . Designing Computational Models of Collaborative Learning Interaction During collaborative learning activities, factors such as students’ prior knowledge, motivation, roles, language, behavior and interaction dynamics interact with each other in unpredictable ways, making it very difficult to predict and measure learning effects. This may be one reason why the focus of collaborative learning research shifted in the nineties from studying group characteristics and products to studying group process. With an interest in having an impact on the group process in modern distance learning environments, the focus has recently shifted again – this time from studying group processes to identifying computational strategies that positively influence group learning. This shift toward mediating and supporting collaborative learners is fundamentally grounded in our understanding of the interaction described by our models of collaborative learning interaction. This workshop explores the advantages, implications, and support possibilities afforded by the various types of computational models of collaborative learning processes. Computational models of collaborative learning interaction provide functional computer-based representations that help us understand, explain, and predict patterns of group behavior. Some help the system automatically identify group members' roles, while others help scientists understand the cognitive processes that underlie collaborative learning activities such as knowledge sharing or cognitive conflict. Some computational models focus specifically on social factors, and may be applied to many different domains, while others are designed to facilitate aspects of task oriented interaction and may be bound to a particular domain. In this workshop, we consider the requirements for modeling different aspects of interaction, exploring issues such as, “What data are needed (e.g. participation statistics, coded dialog, task-based actions) to construct and maintain different types of models?”, and, “How should this data should be represented?” Other open issues include the following:

Representation and Design Considerations

• Many different variables (e.g. participation, dialog acts, task-based actions) can be used to characterize collaborative interaction. Which variables are needed to model which higher-level collaborative learning processes?

• How much and what kind of contextual, domain specific information is needed in the computational model to support collaborative learning activities? What are the benefits gained, and how much generality is lost by including contextual information in a computational model?

• How do models that consider both verbal and nonverbal interactions, or both task and social aspects of group learning change the types of support and guidance that we can offer to collaborative learners?

• Is a unified representation of interaction possible/desirable ? For example, would it be useful to have a "Standard Collaborative Interaction Description Language" (SCIDL)? If so, how should we enhance our existing educational modeling languages with specifications for collaborative learning interaction?

Implementation Considerations • What sort of information should be coded and logged, and at what granularity? • What compilation or abstraction methods are needed to construct a computational model from a logfile

describing the group interaction? • How do conceptual models (in terms of, for example, conflict or constructive argumentation) translate into

computational models that can be represented and analyzed by a computer? • What is the technical cost (time, effort, and computational intensity) of making the theoretical indicators we

deal with operational?

2

Evaluation Considerations • How can we ensure that a computational model accurately reflects the interaction process it is intended to

model? For example, can pedagogical interventions be used to evaluate the efficiency and accuracy of different types of models?

• What techniques enable us to take advantage of student and/or teacher feedback to continually refine and evaluate operational computational models?

• How might we share corpora, and compare and evaluate different models of collaboration across research disciplines?

Related Information Proceedings of the 1st International Workshop on Designing Computational Models of Collaborative Learning Interaction can be found at: http://sra.itc.it/people/soller/CSCL2002/Computational-Models-Workshop.html The co-chairs have also been involved in the organization of the following related workshops:

Analysis and Modelling of Collaborative Learning Interactions at ECAI 2000, Berlin, Germany, August 22, 2000.

User and Group Models for Web-Based Adaptive Collaborative Environments at User Modeling 2003, Johnstown, PA, June 22, 2003.

Artificial Intelligence in Computer-Supported Collaborative Learning at ECAI 2004, Valencia, Spain, August 22-27, 2004.

Workshop Co-Chairs: Amy Soller ITC-IRST 38050 Povo, Trento, Italy Tel: +39 0461 314 358 [email protected]

Patrick Jermann Ecole Polytechnique Fédérale de Lausanne (EPFL) CH-1015 Lausanne, Switzerland Tel : +41 (0)21/693.22.72 [email protected]

Martin Mühlenbrock DFKI - German Research Centre for Artificial Intelligence 66123 Saarbrücken, Germany Tel: +49 681 302 64843 [email protected]

Alejandra Martínez Monés University of Valladolid Department of Computer Science ETSIInformática University of Valladolid, Spain Tel: +34 (9)83 423000 exts. 5623 / 4548 [email protected]

Program Committee: Angeles Constantino González, ITESM Campus Laguna, Mexico Alain Derycke, Université des Sciences et Technologies de Lille, France Pierre Dillenbourg, EPFL, Switzerland Brad Goodman, MITRE, Bedford, MA, USA Katrin Gassner, Fraunhofer ISST, Germany Elena Gaudioso, UNED, Spain Peter Reimann, University of Sydney, Australia Marta Rosatelli, Universidade Católica de Santos, Brazil Ron Stevens, University of California, Los Angeles, USA Julita Vassileva, University of Saskatchewan, Saskatoon, Canada

3

http://sra.itc.it/people/soller/CSCL2002/Computational-Models-Workshop.html

http://collide.informatik.uni-duisburg.de/ecai-2000/

http://www.ia.uned.es/~elena/um03-ws/

http://www.ia.uned.es/~elena/ecai04-ws/ecai04cfp.html

. . . . . .

.. . .

Table of Contents Amy Soller, Patrick Jermann, Martin Mühlenbrock, and Alejandra Martínez Designing computational models of collaborative learning interaction: Introduction to the workshop proceedings 5

Nikolaos Avouris, Meletis Margaritis, and Vassilis Komis Modelling interaction of small-groups synchronous problem-solving activities: The Synergo approach 13

Antonio Carlos da Rocha Costa and Gracaliz Pereira Dimuro The case for using exchange values in the modelling of collaborative learning interaction 19

Brad Goodman, Frank Linton, Guido Zarrella, and Robert Gaimari Predicting trouble during collaborative learning 25

Alejandra Martínez, Luis A. Guerrero, and César A. Collazos A model and a pattern for data collection on collaborative actions in CSCL systems 31

Martin Mühlenbrock A computational model to differentiate between action and interaction in shared workspaces 37

Dan Suthers Implications of shared representations for computational modeling 42

Patrícia Azevedo Tedesco and Marta C. Rosatelli Helping groups become teams: Techniques for acquiring and maintaining group models 53

Julita Vassileva, Ran Cheng, and Lingling Sun Designing mechanisms to stimulate contributions in collaborative systems for sharing course-related materials 59

Marcos Augusto F. Borges, and M. Cecilia C. Baranauskas A process to analyze interactions in collaborative systems with text-based computer mediated communication tools 65

E. J. R. de Castro, G. M. da Nóbrega, E. Ferneda, S. A. Cerri, and F. Lima Towards interaction modelling of asynchronous collaborative model-based learning 71

Katrin Gassner Using patterns to reveal e-mail communication structures 77

Julieta Noguez and Enrique Espinosa Improving learning and soft skills using Project Oriented Learning in software engineering courses 83

Thereza P. P. Padilha, Leandro M. Almeida, and João B. M. Alves Mining techniques for models of collaborative learning 89

Luciane Fadel Simao and Adalberto Nazareth ANIMATED-CHAT: Facial expression to support sense of presence 95

4

Designing Computational Models of Collaborative Learning Interaction: Introduction to the Workshop Proceedings

Amy Soller1, Patrick Jermann2, Martin Mühlenbrock3, Alejandra Martínez4

1 ITC-IRST, via Sommarive 18, 38050 Povo, Trento, Italy

2 Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland 3 German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany

4 Dept. of Computer Science, Universidad de Valladolid, Valladolid, Spain Email: [email protected], [email protected], [email protected], [email protected]

Abstract: Computational models of collaborative learning interaction provide functional computer-based representations that help us understand, explain, and predict patterns of group behavior, and support group learning processes. These models may assist students and teachers in managing and guiding the interaction during learning activities. In this paper, we introduce the proceedings of the 2nd International Workshop on Designing Support for Collaborative Learning Interaction by exploring the advantages, implications, and support possibilities afforded by various types of computational models, in the context of a conceptual framework that we have developed.

Keywords: regulate, mediate, mirroring, metacognitive, guiding, collaboration management

Introduction

During collaborative learning activities, factors such as students’ prior knowledge, motivation, roles, language, behavior, and group dynamics interact with each other in unpredictable ways, making it very difficult to measure and understand learning effects. This may be one reason why the focus of collaborative learning research shifted in the nineties from studying group characteristics and products to studying group process (Dillenbourg, Baker, O’Malley, & Blaye, 1995; Jermann, Soller, & Muehlenbrock, 2001). With an interest in having an impact on the group process in modern distance learning environments, the focus has recently shifted again – this time from studying group processes to identifying computational strategies that positively influence group learning. This shift toward mediating and supporting collaborative learners is fundamentally grounded in our understanding of the group activity described by our models of collaborative learning interaction. The papers included in these workshop proceedings explore the advantages, implications, and support possibilities afforded by the various types of computational models of collaborative learning processes.

Computational models of collaborative learning interaction provide functional computer-based representations that help us understand, explain, and predict patterns of group behavior, and support group learning processes. These models can help us determine how to structure the environment in which the collaboration takes place, or regulate the student interaction during the learning activities (Jermann, Soller, & Lesgold, 2002). We very briefly describe the role of computational models in structuring the group learning environment, and then focus the remainder of our discussion on their role in regulating interaction.

Structuring approaches aim to create favorable conditions for learning by designing or scripting the situation before the interaction begins (Dillenbourg, 2002). For example, we might structure the learning experience by varying the characteristics of the participants, the size and composition of the group, or the definition and distribution

ITS 2004 Workshop on Computational Models of Collaborative Learning

5

of student roles. We might also strategically select a subset of learning tools, activities, and communication media with desired characteristics, or change the appearance of the environment based on the nature of the task (e.g. writing, problem-solving) or the configuration of the group. A computational model, describing students’ prior behavior under similar conditions might be used to strategically construct learning teams and activities, or plan mediation schemes.

Regulation approaches support collaboration by taking actions once the interaction has begun. Interaction regulation is a complex skill that requires a quick appraisal of the situation based on a comparison of the current situation to a model of desired interaction. In the classroom, the regulation of student interaction is performed by a teacher, taking into account complex variables such as the observed student interaction, various experiences from years of teaching, and knowledge of the students’ personalities and typical behaviors. The difficulty in eliciting the knowledge needed to account for these complex variables, and determining the manner and degree to which each contributes to the collaborative learning outcome, has presented significant challenges to the computational modeling, analysis, and assessment of group learning activities. How might a computer assess the quality of knowledge sharing, or measure the degree of constructive conflict between students? It is too early to tell whether or not we will ever be able to offer the supportiveness of a human teacher, online; however, a few research projects have begun to explore the possibilities of enriching CSCL environments with tools to support and enhance collaboration management. In this paper, we describe a few of these tools, in the context of a conceptual framework that we have developed to organize and describe the array of available collaborative support options. Our objective in presenting this conceptual framework is to help structure discussions during this workshop, and provide a focus for the papers in these proceedings. We first describe a general framework that we have found useful for understanding the process of computer-supported collaboration management, highlight a few of the workshop’s themes in the context of this framework, and present open issues for discussion and future research.

Designing support for collaboration management Managing collaborative interaction means supporting group members’ metacognitive activities related to their interaction. It may be facilitated through activities such as providing on-line dynamic feedback to students, or off-line analyses of the students’ collaboration to instructors. The students, instructors, or system might then recommend actions to help students manage their interaction by reassigning roles, addressing conflicts and misunderstandings, or redistributing participants’ tasks, given their levels of expertise.

In this section, we present a framework for describing the process of collaboration management, building upon the work of Jermann, Soller, and Muelhenbrock (2001) and Barros and Verdejo (2000). Collaboration management follows a simple homeostatic process, illustrated in Figure 1, that continuously compares the current state of interaction with a target configuration (the desired state). Pedagogical actions are taken whenever a perturbation arises, in order to bring the system back to equilibrium. Because the definition of the desired state may not be fully known, and may also change during the course of group activity, the framework presented here provides a general description of the activities involved in computer-supported collaboration management, rather than a means for predicting collaborative learning outcomes.

The framework, or collaboration management cycle is represented by a feedback loop, in which the metacognitive or behavioral change resulting from each cycle is evaluated in the cycle that follows. Such feedback loops can be organized in hierarchies to describe behavior at different levels of granularity (e.g. operations, actions, and activities). The collaboration management cycle is defined by the following phases:

• Phase 1: The data collection phase involves observing and recording the interaction. Typically, users’ actions (e.g. ‘user1 clicked on I agree’, ‘user1 changed a parameter’, ‘user1 created a text node’) are logged and stored for later processing.

• Phase 2: The next phase involves selecting and computing one or more higher-level variables, termed indicators, to represent the current state of interaction. For example, an agreement indicator might be derived by comparing the problem solving actions of two or more students, or a symmetry indicator might result from a comparison of participation indicators.


6

Collect &Aggregate

Interaction Data

Phases 1 & 2Compare Current

State of Interactionto Desired State

Phase 3

CurrentCurrentState ofState of

InteractionInteraction

Offer Advice and Guidance

Phase 4

MirroringTools

Meta-CognitiveTools

Guiding Systems

DesiredDesiredState ofState of

InteractionInteraction

Figure 1. The Collaboration Management Cycle

• Phase 3: The interaction can then be “diagnosed” by comparing the current state of interaction to a desired model of interaction. We define the desired model as a set of indicator values that describe productive and unproductive interaction states. For instance, we might want learners to be verbose (i.e. to attain a high value on a verbosity indicator), to interact frequently (i.e. maintain a high value on a reciprocity indicator), and participate equally (i.e. to minimize the value on an asymmetry indicator).

• Phase 4: Finally, if there are discrepancies between the current state of interaction (as described by the indicator values) and the desired state of interaction, some remedial actions might be proposed. Simple remedial actions (e.g. ‘Try letting your partner have control for a while’) might result from analyzing a model containing only one indicator (e.g. word or action count), which can be directly computed from the data, whereas more complex remedial actions (e.g. ‘Try explaining the concept of generalization to your partner using a common analogy’) might require more sophisticated computational analysis.

Phase 4 is not the final phase in this process. Remediation by the system or human instructor will have an impact on the students’ future interaction, and this impact should be re-evaluated to ensure that it produced the desired effects. The arrows that run from phase 4 back through the illustration representing the logging of learners’ actions, to phase 1 indicates the cyclic nature of the collaboration management cycle, and the importance of evaluation and reassessment at the diagnostic level.

Understanding the locus of processing and computer-based support options Research in distributed cognition suggests that cognitive and metacognitive processes might be spread out and shared among actors in a system, where these actors may constitute both people and tools (Hutchins, 1995; Salomon, 1993). Following this idea, computers might offer support for any or all of the four phases described in the previous section. The locus of processing describes the location at which decisions are made about the quality of the student interaction, and how to facilitate this interaction. Depending on the requirements and goals of the learning activity, the locus of processing may be located anywhere on a continuum between the system, instructors, and collaborating students. For example, a teacher, or the group members themselves, might observe the interaction, compare its current state with implicit or explicitly agreed upon referents, and propose changes to the communicative rules or


7

division of labor. In this case, the locus of processing is in human hands. Alternatively, parts of this process might be managed by a computer system, thereby shifting the locus of processing towards the computer.

Systems that collect interaction data and construct visualizations of this data tend to place the locus of processing at the user level, whereas systems that advise and coach aggregate and process this information directly. In the remainder of this section, we describe three computer-based support options that arise when the computer takes over various phases of the collaboration management process presented in the previous section.

Mirroring Tools

Metacognitive Tools

Guiding Systems

Mirroring Tools

Metacognitive Tools

Guiding Systems

Mirroring tools automatically collect and aggregate data about the students’ interaction (phases 1 and 2 in Figure 1), and reflect this information back to the user, for example, as graphical visualizations of student actions or chat contributions. These systems are designed to raise students’ awareness about their actions and behaviors. They place the locus of processing in the hands of the learners or teachers, who must compare the reflected information to their own models of desired interaction to determine what remedial actions are needed.

Metacognitive tools display information about what the desired interaction might look like alongside a visualization of the current state of indicators (phases 1, 2 and 3 in Figure 1). These systems provide the referents needed by the learners or human coaches to diagnose the interaction. Like mirroring tools, users of metacognitive support tools are responsible for making decisions regarding diagnosis and remediation.

Guiding systems perform all the phases in the collaboration management process, and propose remedial actions to help the learners. The desired model of interaction and the system’s assessment of the current state are typically hidden from the students. The system uses this information to make decisions about how to moderate the group’s interaction.

Fundamentally, these three approaches rely on the same model of interaction regulation, in that first data is collected, then indicators are computed to build a model of interaction that represents the current state, and finally, some decisions are made about how to proceed based on a comparison of the current state with some desired state. The difference between the three approaches above lies in the locus of processing. Systems that collect interaction data and construct visualizations of this data place the locus of processing at the user level, whereas systems that offer advice process this information, taking over the diagnosis of the situation and offering guidance as the output. In the latter case, the locus of processing is entirely on the system side.

Selecting and designing the most appropriate computational approach for supporting group interaction means evaluating the learners’ needs and assessing the available computational resources. Each of the three support options described in this section presents different advantages and disadvantages (described in more detail in the next section, and throughout these proceedings), and many combinations of approaches can be complementary. For example, imagine a system that progressively moves the locus of processing from the system side to the learner side: a guiding tool that becomes a metacognitive tool and finally a mirroring tool. As students observe the methods and standards that the system uses to assess the quality of the interaction, they might develop a better understanding of the system’s process of diagnosis, allowing the responsibility for interaction regulation to be progressively handed over to the students. Once the students have understood (internalized) these standards, simply displaying the indicators in a mirroring tool might be sufficient.


8

Workshop highlights from a collaboration management perspective In this section, we highlight a few of the themes in these workshop proceedings with regard to the collaboration management cycle.

Phase 1

The first phase in each cycle is the data collection phase, in which the student interaction is recorded to a logfile, database, or internal data cache. While this step might seem simple, it still poses significant challenges. On one hand, the establishment of a standard data format might enable researchers to share and reuse analysis tools across different CSCL systems. On the other, such standards may limit the level of customization we can provide to users, who may want to choose specific combinations of data to analyze. Regarding the first issue, we do not know of a widely accepted data format that could facilitate the reuse and sharing of analysis methods developed by different researchers; however, some of the papers in these proceedings present contributions in this direction.

For example, Avouris, Margaritis, and Komis propose a system, Synergo, that builds on the Object-oriented Collaboration Analysis Framework (OCAF). Their system is unique in that student interaction and workspace actions are analyzed from the shared objects’ point-of-view. The objects that students manipulate independently compile statistics on their use, and contribute to the definition of indicators describing their owners’ collaborative behavior. OCAF includes a formal definition of events as tuples that include time, actions, objects, and configurable types of events. These elements are also considered in the collaborative action model, proposed by Martínez, Guerrero, and Collazos, that includes aspects related to the context of the interaction (Martinez, Dimitriadis, & de la Fuente, 2003). Drawing on this model of collaborative action, the authors address the problem of customizing the collection of data through the use of the command design pattern for the implementation of the data collection, modularizing it and enabling the desired customization. This design pattern is a general solution that can be used to define logging functionality in any type of application (not only those that are designed to mediate collaborative activity). Castro and colleagues present a similar architecture based on the Model-View-Controller design pattern, designed to support collaborative model construction.

Phase 2

The second phase involves computing one or more higher-level variables, termed indicators, to represent the current state of interaction. Information about the interaction takes many forms, from low-level user interface events (e.g. mouse clicks or movements, drag & drop actions, keystrokes) to actions that carry meaning in terms of the task (e.g. message posts, utterance type or category selections, graph node or edge deletions). The aggregation process carried out in this phase may lead to cognitive interpretations of certain data combinations, enabling researchers to assign value to these aggregated actions in terms of learning or problem solving (e.g. propose counter-argument, refine simulation setting, complement schema, explain strategy).

Some of the systems presented at the workshop implement indicators that aim at helping their users (teachers or students) understand the state of the collaboration specifically in terms of their indicators. For example, Avouris, Margaritis, and Komis propose the collaboration factor (CF), as an aggregation of other, quantitative, lower-level indicators, and Padihla, Almeida, and Alves propose performance reports based on a set of quantitative and qualitative indicators. Both of these indicators are graphically displayed on the time axis, facilitating the analysis of collaboration over a set time period. Vassileva, Cheng, Sun, and Han’s system aggregates the different types of contributions users have made to a virtual community over time, and shows users visualizations of their membership level in the community. The possibility of achieving a silver or gold membership status encourages users to contribute to their community.

Phase 3

During the third phase of the collaboration management cycle, the interaction is “diagnosed” by comparing the current state of interaction to a desired model of interaction. The main challenges present during this step are (a) defining, as best possible, the model of desired interaction, and (b) designing algorithms that measure the degree to which the current model of interaction meets the requirements of the desired model, which may be uncertain or unstable.


9

In preparation for this phase, the derivation of indicators from raw data during the second phase might benefit from the action or dialog classification techniques described by Tedesco and Rosatelli. For instance, plan recognition techniques might be used to identify and distinguish different problem solving strategies, Markov modeling might be used to recognize or predict interaction patterns (e.g. Soller & Lesgold, 2003), pattern recognition techniques might allow sequences of events to be grouped into more general behavioral units (as in Gassner’s contribution), Bayesian modeling might be useful for describing the relationships between actions and their causal effects (as in Muehlenbrock’s contribution), and filtering techniques might help in determining which actions are meaningful, and which should be disregarded as “noise” (e.g. rearranging the nodes in a graph for aesthetic reasons).

Although these learning and classification techniques help in understanding, modeling, and assessing student activity, the problem of analyzing unstructured student dialogue is still an open issue. Padihla, Almeida, and Alves discuss how various text and data mining techniques might address this issue, and Goodman, Linton, Zarrella, and Gaimari present a specific example in which machine learning methods are used to train a system to recognize when students are experiencing trouble related to specific aspects of interaction. Their approach involves training neural networks with segmented, coded (speech act) student dialog and surface features (e.g. question marks and keywords).

Costa and Dimuro propose an alternative approach to diagnosing the state of interaction, based on the notion of social exchange values, as defined by Piaget and Homans. Their contribution explains the concept of exchange values, and how they can be used to help assess the quality of the interaction. Underlying this approach is the idea that effective social exchange should tend towards equilibrium. Suthers also presents an interesting proposal, building on Muehlenbrock and Hoppe’s (1999) analysis of actions in shared workspaces, in which collaborative learning interaction patterns might be assessed by modeling mode shifts between domain-oriented verbal and task actions.

Phase 4

If there are discrepancies between the current state of interaction (as described by the indicator values) and the desired state of interaction, the system may enter the fourth phase, in which it alerts a human facilitator as to the nature of the discovered discrepancies, or directly takes remedial actions in the collaborative virtual space. For example, the system described by Borges and Baranauskas alerts a facilitator when it detects critical periods in the student interaction, recommending intervention.

Tedesco and Rosatelli review several different systems in which a computer-based coach provides guidance to the learning group. For example, the COLER system (Constantino-Gonzalez, Suthers, & Escamilla de los Santos, 2002) detects differences between the students’ personal and shared workspaces, and differences between students’ participation levels in order to identify opportunities for facilitating group learning interactions. Tedesco’s MarCo system models and monitors the group dialog, intervening with recommendations when it detects meta-cognitive conflicts.

In some cases, metacognitive tools that monitor the state of interaction are not all that different from systems that provide advice. For example, suggesting that a student participate more does not require much more computation than displaying students’ participation statistics; moreover both approaches may have the same effect. These systems begin to differ when the knowledge behind the indicators requires a great enough level of inferencing to warrant having a coach analyze the data to scaffold the learning process.

Open Issues In these workshop proceedings, we see how Artificial Intelligence techniques such as pattern and plan recognition, and data mining may be valuable in the construction of indicators from raw interaction data, and how guiding systems might diagnose interaction, proposing recommendations to learners or teachers. The theoretical and experimental foundations for our models, however, must be strengthened, justified, and assessed. What does it mean when we calibrate a set of indicators to constitute a model of desired interaction, and what learning theories or experimental results allow for this calibration? This leads us to the broader issue of how to quantify and translate well-known theories from the learning and cognitive sciences into computational models that can be used to diagnose student interaction. For example, how might the principle relating elaborated explanations to learning gains (Webb, 1992) be quantified as a set of calibrated indicators that can be computed on the fly during computer-mediated


10

interaction? A “sufficiently elaborated explanation” might be relatively long, and refer to several domain concepts, making computer diagnosis difficult.

The techniques and systems described throughout these proceedings use different standards for diagnosis. How might we develop modular, reusable solutions that would allow researchers to share and reuse tools in different CSCL environments? Instead of proposing new data formats and interfaces, would it be reasonable to tackle this problem in parallel with current efforts toward introducing collaboration aspects in e-learning standards?

In the future, we hope to develop reusable models of collaborative processes, based on modular architectures, that can provide the computational, theoretical, and pedagogical foundations for guiding tools, while encouraging metacognitive reflection by both teachers and students. Such models might even be used in teacher training, to help explain breakdowns in student interaction, or the dynamics of productive collaborative learning interaction.

Many of the approaches presented in these proceedings address effects with technology, rather than effects of technology (Kolodner & Guzdial, 1996; Salomon, Perkins & Globerson, 1991). Effects with technology refer to the changes in the group dynamics that are triggered by software tools, whereas effects of technology refer to the outcome of the collaboration, both for the individual and the collective group. These outcomes include the skills that students acquire or improve, and whether or not these skills might transfer to a new learning situation or group experience. More research is needed to determine how visual feedback through mirroring and metacognitive tools, or advice from guiding systems can lead to learning gains. In designing support for the collaborative learning process, we must not forget to assess the product.

References Barros, B., & Verdejo, M.F. (2000). Analysing student interaction processes in order to improve collaboration. The

DEGREE approach. International Journal of Artificial Intelligence in Education, 11, 221-241.

Constantino-Gonzalez, M.A., Suthers, D.D. and Escamilla de los Santos, J.G. (2002). Coaching web-based collaborative learning based on problem solution differences and participation. International Journal of Artificial Intelligence in Education, 13, 263-299.

Dillenbourg, P., Baker, M., Blaye, A. & O'Malley, C. (1995). The evolution of research on collaborative learning. In P. Reimann & H. Spada (Eds.) Learning in Humans and Machines: Towards an Interdisciplinary Learning Science (pp. 189-211). Oxford: Elsevier.

Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed). Three worlds of CSCL. Can we support CSCL (pp. 61-91). Heerlen, Open Universiteit Nederland

Hutchins (1995). How a cockpit remembers its speeds. Cognitive Science, 19, 265-288.

Jermann, P., Soller, A., & Lesgold, A. (2004). Computer software support for CSCL. In P. Dillenbourg (Series Ed.) & J. W. Strijbos, P. A. Kirschner & R. L. Martens (Vol. Eds.), Computer-supported collaborative learning: Vol 3. What we know about CSCL ... and implementing it in higher education (pp. 141-166). Boston, MA: Kluwer Academic Publishers.

Jermann, P., Soller, A., & Muehlenbrock, M. (2001). From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning, Maastricht, The Netherlands, 324-331.

Kolodner, J., & Guzdial, M. (1996). Effects with and of CSCL: Tracking learning in a new paradigm. In T. Koschmann (Ed.) CSCL: Theory and Practice of an Emerging Paradigm (pp. 307-320). Mahwah NJ: Lawrence Erlbaum Associates.

Martínez, A., Dimitriadis, Y., de la Fuente, P. (2003). Towards an XML-based model for the representation of collaborative action. Proceedings of Computer Support for Collaborative Learning (CSCL 2003), Bergen, Norway, 379-388.


11

Mühlenbrock, M., & Hoppe, U. (1999). Computer supported interaction analysis of group problem solving. In C. Hoadley & J. Roschelle (Eds.) Proceedings of the Conference on Computer Supported Collaborative Learning (CSCL-99) (pp. 398-405). Mahwah, NJ: Erlbaum.

Salomon, G. (1993). No distribution without individual’s cognition: a dynamic interactional view. In G. Salomon (Ed.) Distributed cognitions. Psychological and educational considerations (pp. 111-138). Cambridge University Press.

Salomon, G., Perkins, D., & Globerson, T. (1991). Partners in cognition: Extending human intelligence with intelligent technologies. Educational Researcher, 20(4), 2-9.

Soller, A., & Lesgold, A. (2003). A computational approach to analyzing online knowledge sharing interaction. Proceedings of Artificial Intelligence in Education 2003, Sydney, Australia, 253-260.

Webb, N. (1992). Testing a theoretical model of student interaction and learning in small groups. In R. Hertz-Lazarowitz and N. Miller (Eds.), Interaction in Cooperative Groups: The Theoretical Anatomy of Group Learning (pp. 102-119). New York: Cambridge University Press.


12

Modelling interaction during small-groups synchronous problem-solving activities: The Synergo approach

Nikolaos Avouris, Meletis Margaritis, Vassilis Komis

University of Patras, Patras, Greece { N.Avouris, Margaritis }@ee.upatras.gr, [email protected]

Abstract Monitoring and analysis of activities of small groups of students - collocated or at a distance- engaged in synchronous collaborative problem solving activity is the subject of this paper. This is discussed in the frame of Synergo, a new synchronous collaboration support environment that monitors the activity and permits visualization of various quantitative parameters, like density of interaction, symmetry of partners’ activity, degree of collaboration etc, particularly useful for understanding the mechanics of collaboration. Synergo has been used for synchronous building of flow charts, concept maps, entity-relation diagrams and other semantic modeling activities by small groups of students and has been proposed as a testbed for micro-analysis of small scale interaction in order to gain an insight in collaborative learning. Keywords: Synchronous collaborative problem solving, analysis of collaborative activity, collaboration factor Introduction Socially inspired theories, supported by the growing development of network and collaborative technology, have increased research on technology-based collaborative problem solving environments. These theories usually influence our considerations on effectiveness of the collaborative problem solving process, as well as the design of the collaboration-support tools involved. According to these perspectives, the methodological issues of collaboration analysis are of prime importance, given that they are directly related to the development of this research and technology area and the common understanding of the various disciplines involved. In problem-solving collaborative learning activities in which the computer environment constitutes in itself a mediational resource, it contributes to the creation of a shared referent between the social partners (Rochelle et al, 1995). Typically the direct manipulation environments used are characterised by actions on objects representing entities or on concepts meaningful to the users. Usually operations on these objects have a reversible incremental effect on the ‘environment’ represented on a shared computer screen. Often more than one actor interact directly or indirectly with the objects in this world modifying their state, communicating between them and through the objects, as they advance problem solution. Various methods have been proposed for modelling and analysis of interaction during collaborative problem solving (e.g. Jermann et al. 2001, Muelenbrock and Hoppe, 1999, Martinez et al. 2003) In this paper we outline an innovative framework for analysis of collaborative problem solving activities. This framework has been used for conceptualization of the situation of groups of individuals engaged in exploratory and design problem solving activities and for evaluation of the effectiveness of IT design supporting the process. This methodological framework is based on the “Object-oriented Collaboration Analysis Framework (OCAF)”, originally proposed by Avouris et al. (2002, 2003). Recently, analysis tools have been built to support this framework, while OCAF has been used in a number of field studies investigating various aspects of collaborative problem solving (e.g. Komis et al. 2002, Margaritis et al. 2003, Avouris et al. 2004). In this paper we discuss the collaboration-support environment and the analysis method and tools that have been recently built to support the framework.


13

OCAF studies the activity through the objects of the solution, that is the objects that exist in the problem-solving context. These objects become the centre of attention and are studied as entities that carry their own history and are “acted upon” by their owners. This perspective produces a new view of the process, according to which the solution is made up of structural components that are “owned” by actors who have contributed in various degrees to their existence. This view of the world, can be useful, as it reveals the contribution of the various actors in parts of the solution, and the relevant focus shifts (Bertelsen and Bodker, 2003), identifies areas of intense collaboration in relation to the final solution and can relate to other analysis frameworks like interaction analysis. In this paper, we describe first the Synergo collaborative problem-solving environment. Subsequently, an outline of the model of interaction is included together with presentation of the functionality of the tools that have been proposed to support analysis of interaction. Through the Synergo analysis tools, the researcher can playback the activity off-line and annotate the activity and the produced solution using an annotation scheme which can be defined and adapted according to the specific objectives of the study. A brief example of use of the framework and the tools in collaborative problem-solving situations is also presented. The Synergo Environment Synergo is a new collaboration support environment based on the Abstract Collaborative Applications Building Framework (ACABF), also used for building ModellingSpace (Margaritis et al. 2003) and ModelsCreator v3 (Fidas et al. 2002). Synergo architecture supports synchronous collaboration, as well as integration of collaboration analysis and visualization tools.

Figure 1. The Synergo environment: client user interface

Shared activity space

Chat tool Libraries of

primitive objects

The Synergo environment (http://www.ee.upatras.gr/hci/synergo) is a client-server distributed application, which comprises a suite of interconnected tools to support collaborative drawing activities. The main functionality of the Synergo environment is described through fig. 1, which shows a typical problem-solving activity. Synergo supports building of different kinds of diagrams. It contains libraries for building flowcharts, entity-relationship diagrams, concept maps, data flow diagrams etc. On the left-hand side column of figure 1, libraries of primitive objects are shown. The activity is monitored and logfiles are generated and made available for inspection by the users or researchers. Synchronous collaboration for problem solving is a case of computer-supported collaboration based on the concept of shared artefact/work surface (Dix et al, 1998). The related notion of feed-through the artefact implies that one participant's manipulation of shared objects can be observed by the other participants. This communication through the artefact can be as important as direct communication between participants. Considering that the collaborative activity is done mainly between partners at a distance, the direct communication mechanism has to be defined. A text communication has been used in this case. One additional decision is related to the design of the shared activity space. In Synergo a mixture of alternatives is provided. A strict WYSIWIS (what you see is what I see) is allowed in the shared problem-solving window. This is because most of communication and reasoning is based on this shared viewpoint, which becomes the main grounding mechanism of dialogue and through which eventually common understanding can occur. However all additional operations outside this shared workspace, e.g. relating to browsing of activity sheets and other auxiliary material, saving of the flow chart or using private activity windows, should be performed independently by partners involved, a model-level coupling approach according to Suthers (2001). This approach, also known as relaxed


14

WISIWYS, guarantees only that users will see the same semantic state of a shared model, but the views may be entirely different. In Synergo a floor control coordination mechanism is included. This mechanism involves the notion of the Action Enabling Key, which is owned by one of the participants at any given time. This key owner can then act in the shared workspace, while the rest just observe this activity and make comments through the chat tool. This mechanism is supported by key request, key accept, key pass, key reject functions, which can be found in the Coordination Panel (see fig.1). Experiments with this floor control mechanism, see also (Fidas et al. 2001) and (Komis et al. 2002), demonstrate that it supports reasoning about action, as partners need to reason and negotiate during key requests. Synergo users may opt for this mechanism or may decide to act in the shared activity space with no specific floor control, in which case locking is effected at the level of the single entity. In the frame of the collaborative use of Synergo, a dialogue tool has been integrated, shown at the right panel of fig.1, which is based on an instant messaging protocol, using the same point-to-point connection and protocol of the shared activity space. Through this, text messages are exchanged during collaborative problem solving. The chat tool is activated from the collaboration panel. The possibility of definition of dialogue openers is included in this tool, as shown in figure 2, however due to concerns related to the usability of such approaches in the case studies discussed here, such dialogue openers have not been used.

Figure 2. Examples of dialogue openers of the chat tool Other means for exchange of text messages are the sticky notes as text containers positioned in the activity space, associated to existing objects, through which, gestures to them can be simulated. An innovative feature of Synergo relates to analysis of collaboration activities. So a number of Analysis and Visualization tools are included in the environment. These are mainly used by the teachers and researchers, while limited versions of the tools may be used in some cases by students as meta-cognitive aids, as is the case of the level of collaboration monitoring display. The main functionality of the Analysis tool is the presentation and processing of logfiles, which are created during Synergo use. These logfiles contain actions and text messages of all partners, in sequential order. The logfiles are based on the format of the coordination and communication protocol and are stored in XML. These files can be viewed, commended and annotated by the researchers, using an adequate analysis framework, as discussed by Avouris et al. (2003a). A related functionality is the capability of the analysis environment of posterior reproduction of the modelling activity, using this logfile, in a step-by-step or continuous way. This is complementary to the logfile inspection and annotation functionality. Modelling Collaboration In this section we describe the key parameters through which we can model collaborative problem solving activity in Synergo. We suppose that the activity involves a small group of subjects (actors) who are engaged in collaborative problem solving (2 to 5 actors). Problem solving activity is usually considered as a process of refinement of abstract ideas (“abstract objects”) and externalisation of these ideas in the form of parts of the solution to the given problem. Collaborative activity is based on communication, which takes the form of either direct communication acts or operations in the shared activity space. The activity is defined according to the following four dimensions: − The time dimension − The actors’ dimension: actors, . { }kAAAA ,...,, 21=− The objects’ dimension: . In the frame of the Synergo tool, a solution is considered as made

of components (objects that compose the final solution), rejected components and abstract components { lOOOO ,...,, 21= }

− The typology of event dimension: This is a dimension through which interpretation of the activity can take


15

place. We assume that there is an existing analytical framework, which defines this typology. If r is the finite number of expected event types, then we define a set { }rTTTT ,...,, 21= as the analytical framework of the study. While in the original OCAF proposal such a closed set T was included, (Avouris et al. 2003), in Synergo, we consider the method as independent of a specific analytical framework, so set T can be defined by the framework user.

Using the above four dimensions we can describe any given activity as a set of discrete non-trivial events produced by the actors. These define an ordered set of m events { }mEEEE ,...,, 21= . Each one of these events is related to meaningful operations of the actors who interact with objects of set O. Each event is defined as a tuple

where , t the event timestamp, A the actor who performed the operation of the specific event, O an optional parameter referring to the object of the specific operation and T an optional parameter which interprets the event according to the analysis framework T.

( )iTOAti TOAtEtAOT

][],[,,= ],1[ mi ∈

This is a useful model for ethnographic studies. Every time an event is produced by the actors, this is recorded and a history of such events, i.e. an ordered list of Es can be produced, as a result of such an activity. No mental or cognitive operators are defined, as these can be generated later as interpretations of the recorded activity. This model permits further analysis and interpretation of the activity, while quantitative indices of the activity can be easily produced or visualizations can be generated (Margaritis et al. 2004), as discussed in the next section. Synergo adheres to a typology of generated events, thus automating the task of categorization of observed events (insertion, modification, deletion of primitive objects in the shared space and exchange of text messages), every time such an operation is recorded, this is automatically categorized according to the scheme of analysis defined by the user. OCAF suggests interpretation of exchanged messages (written dialogues during collaboration by distance), or recorded oral utterances (during face to face collaboration), in relation to operations towards “objects” of the activity space, using a language for action approach (Winograd 1987), defining a unifying framework for analysis of dialogue and action.

Quantitative indices of collaboration Using the model of activity described above, a number of indices have been defined and accordingly presented in a visual form. Some of these indices relate to the density of occurrence of a type of event per time interval tq, e.g. number of exchanged text messages per tq, number of new objects in the shared space per tq, etc. One other kind of index is related to the degree of symmetry of activity in the group members. This index describes the relative contribution of the group members in a specific type of events. An example of an empirical index, called Collaboration Factor is described here. For instance, if we assume that N events of Actor A concern object O, then the contribution of Actor A to object O is measured as ( )∑

=

⋅=N

iiAO TWAWAC

1)( , where W(Α) is the relative weight of actor A και

W(T ) is the weight of type T of event i, that contributed to O history. The history factor HF of Ο, is defined as i i ( )

kMACstdevHFO −=1 , where and M is the mean value of the AC for object O. HF takes value around 1 when

there is symmetrical contribution of all actors in the history of object O and around 0 when the object has been discussed and used by small section of the group. The collaboration factor of object O is defined subsequently, as

]1,0[∈HF

mOEL

WHFCF OOOO

)(⋅⋅= , , where W the relative weight of object O in the model, is the length of

action events of object O and m the total number of action events in E. Finally the collaboration factor of the modeling activity CF is defined as the mean value of all components’ collaboration factors, including the abstract objects, or objects

that were discussed and later rejected:

]1,0[∈OCF o )( OOEL

l

l

∑== 1i

OiCF

CF , ]1,0[∈CF .

This parameter, in addition to other indices like the density of activity of specific type of action events per time unit, can produce views of the activity that can lead to understanding of the collaboration dynamics, as discussed in the following section. A case study of analysis of collaboration with Synergo In this section we describe an example of a study that involved analysis of collaborative activity using the Synergo tool. The activity involved building of a concept map of an Internet service (an electronic bookshop was chosen as the example of the service to be model by the participants in this case) by small groups of


16

students of an undergraduate University course, in the frame of one lab session (45’). We focus on one of these groups made of 4 students in this section. The logfile of the activity of this specific group was studied using the Synergo. More details of this study can be found in Avouris et al. (2004). First the relative weights of the activity types and the actors were defined, as seen in figure 3(a). In our case events related to creation and modification of sticky notes are assigned lower weight (0.3), as they are used for administration issues.

Abstract objects

Dialogue messages

Model objects

Deleted objects

Figure 3. (a) Definition of activity type W(T) and actors weights W(A) and

(b) annotation of dialogue events

Subsequently the dialogue events were annotated according to the defined typology. This phase involved definition of abstract entities that appeared in the dialogue. The dialogue annotation window is shown in figure 3(b). Three types of objects are shown in this window: the components of the final solution in the main panel (model objects), the deleted components in the vertical panel and the abstract components at the bottom panel. In the example of fig.3(b) a dialogue event is associated to the abstract object “Amazon model”: Actor Ges said: ”what to assign to the Amazon site?”, This dialogue message was categorized as a Q (Query) and was associated to the abstract object “Amazon model”, by a simple drag operation.

Fi

Aftthe actologfof “grocomanddeggrap


(a)

gure 4. Visualization of collaboration indices (a) Collaboration Factor, (b) Evolution of Actor

)

(b)

(a)

er annotating dialogue events, we are able to playback the activity and produce in numeric and vevolution of the Collaboration Factor. This is shown in figure 4(a). Some other indices, like the rs activity of various types in the shared activity space, can be produced automatically, from thile. Also the contribution of each actor in the activity can be visualized. In figure 4(b) the actor coinsert object” events and chat messages is shown. Each line of these diagrams represents one o

up members. From this picture, it is deduced that the second actor shows relatively low activplex indices like the Collaboration Factor discussed here, are produced as a result of interpretation

dialogue events. An example is the visual representation shown in fig.4(a). This provides an indicaree of collaboration of the group of the four students as they are building the e-shop concept map.h it seems that while for the first period of the activity the degree of collaboration was high, sub

17

(b

activity

isual form density of e Synergo ntribution f the four ity. More

of actions tion of the From this sequently

the partners became more individualistic, working on parts of the solution, as also shown in the annotated concept map of fig 4(a). Later on towards the end of the session, there is more interaction, at the level of specific concepts and entities, the final value was CF=7,32%. Conclusions The innovative nature of Synergo is related to its capacity of monitoring and visualizing activity both of action and dialogue events using a unified framework, implementing the OCAF analysis framework. Dialogue events are assumed to be related to abstract of concrete objects of the solution. Thus the notion of history of objects creation is defined. The researcher using Synergo can define the analysis scheme in terms of types of events, and their relative weight. Also weights are associated to specific group members, so for instance the tutors as members of a group can be assigned with different weights than the students. A number of quantitative indices are calculated by the tool and can be visualized during playback of the activity. Also an intuitive environment for annotation of dialogue events is included which permits categorization of the exchanged messages according to the defined typology and association of them to objects of the solution. The proposed model of interaction in Synergo is used for visualization of indices and support of actors and analysis. No attempt has been made to relate this model to automatic supporting and scaffolding of interaction, as these approaches usually move the locus of control of activity from the user to the system, reducing usability and acceptability of the environment. The Synergo tool has already been effectively used for analysis of interaction of collocated small groups of students (Voyiatzaki et al. 2004) and of distant groups in the context of a course of distant learning (Xenos et al. 2004). It is believed that this kind of environment can facilitate and advance our understanding of the mechanics of collaboration of small groups of students, as micro-scale patterns of interaction and solution building can emerge. This understanding can facilitate support of the activity by tutors or by the environment itself at run time. References Avouris N, Komis V., Margaritis M., Fidas K., (2004a), ModellingSpace: A tool for synchronous collaborative problem solving, Proc. AACE ED-Media, pp. 381–386, Lugano, June 2004. Avouris N., M. Margaritis, V. Komis, (2004b). The effect of group size in synchronous collaborative problem solving activities, Proc. ED Media AACE Conf., pp. 4303-4306, Lugano, June 2004. Avouris N., V. Komis, M. Margaritis, G. Fiotakis, (2004c) An environment for studying collaborative learning activities, Journal of Technology & Society, 7 (2), pp. 34-41, April 2004 . Avouris N.M., Dimitracopoulou A., Komis V., (2003), On analysis of collaborative problem solving: An object-oriented approach, Computers in Human Behavior, 19, (2), March 2003, pp. 147-167. Bertelsen O.W., Bodker S., (2003), Activity Theory, in J. M Carroll (ed.), HCI Models, Theories and Frameworks, Morgan Kaufmann, 2003. Dix A., Finlay J., Abowd G, Beale R., (1998), Human-Computer Interaction, Prentice Hall. Fidas C., Komis V., Tzanavaris S., Avouris N., (2004), Heterogeneity of learning material in synchronous computer-supported collaborative modeling, Computers & Education, (in press). Jermann, P., Soller A. & Muhlenbrock M. (2001) "From mirroring to guiding: a review of the state of the art technology or supporting collaborative learning". In Proceedings EuroCSCL΄2001, Maastrich pp. 324-331. Komis V., Avouris N., Fidas C., (2002), Computer-Supported Collaborative Concept Mapping: Study of Synchronous Peer Interaction, Education and Information Technologies, 7:2, 169–188. Margaritis M., Avouris N., Komis V., (2004), Μethods and Tools for representation of Collaborative Learning activities. Proc. ETPE 2004, September 2004, Athens. Martinez A., Dimitriadis Y., Gomez E., Rubia B., De la Fuente P., (2003), Combining qualitative and social network analysis for the study of classroom social interactions, Computers and Education, 41, (4), pp. 353-368 Muelenbrock, M. & Hoppe, U. (1999), Computer Supported Interaction Analysis of Group Problem Solving. C.Hoadley & J.Roschelle (Eds). In Proc. CSCL 1999; Dec 12-15; Stanford University, Palo Alto, California. Mahwah, NJ: Lawrence Erlbaum Associates; pp. 398-405. Suthers D. 2001, Architectures for Computer Supported Collaborative Learning. In proceedings of the IEEE International Conf. on Advanced Learning Technologies (ICALT2001), 6-8- Aug. 2001. Madison, Wisconsin. Voyiatzaki E., Christakoudis C., Margaritis M., Avouris N., (2004), Algorithms Teaching in Secondary Education: A collaborative Approach, Proc. ED- Media 2004, pp. 2781-2789, Lugano, June 2004. Winograd T., (1987). A Language/Action Perspective on the Design of Cooperative Work, Human-Computer Interaction 3:1 (1987-88), 3-30. Xenos M., Avouris N., Komis V., Stavrinoudis D., Margaritis M., (2004), Synchronous Collaboration in Distance Education: A Case Study on a CS Course, Proc. IEEE ICALT 2004, Joensuu, FI.


18

The Case for Using Exchange Values in the Modelling ofCollaborative Learning Interaction?

Antonio Carlos da Rocha Costa12 and Gracaliz Pereira Dimuro1

1 Escola de Informatica – Universidade Catolica de Pelotas (UCPel)Pelotas, RS, Brazil.

2 PPGC-PGIE – Universidade Federal do Rio Grande do Sul (UFRGS)Porto Alegre, RS, Brazil.

{rocha,liz}@atlas.ucpel.tche.brhttp://gmc.ucpel.tche.br/valores

Abstract. This paper presents the case for the use of social exchange values as an impor-tant element in the modelling of collaborative learning interaction. The concept of exchangevalues is explained, and two independent, fundamental approaches to exchange values aresummarized, namely, the approaches of Jean Piaget and George Homans. An argument ismade for the complementarity of those theories. The roles exchange values and the relatedconcept of equilibrium of social exchanges may play in the modelling of learning interac-tions are explained. The paper concludes with tentative answers for the main questionsposed to the Workshop, in what they refer to exchange values applied to the modelling ofcollaborative learning interaction: representation and design considerations, implementationconsiderations, and evaluation considerations.

1 Introduction

This paper presents the case for the use of social exchange values as an important tool in the mod-elling of collaborative learning interaction, pointing toward the need for tools that take exchangevalues into account when developing strategies to influence group learning.

The paper is structured as follows. In Section 2, we summarize two fundamental theories ofsocial exchange values, one by Jean Piaget, another by George Homans. We also explain how wethink the two theories can fit together. In Section 3, we summarize our efforts to develop a com-putational model for exchange values, and the applications we have sketched for such values in thecontext of computational systems. The roles exchange values and the related concept of equilib-rium of social exchanges may play in the modelling of learning interactions are sketched. Finally,in Section 4, based on such works, we consider the possibility of applying exchange values to themodelling of interactions in collaborative learning environments, and try to give tentative answersfor the main questions of the Workshop, in that respect: representation and design considerations,implementation considerations, and evaluation considerations. Section 5 brings the Conclusion.

2 A Summary of Piaget’s and Homans’ Theories of Exchange Values

Jean Piaget (in the 1940’s) and George Homans (in the 1950’s) independently developed twotheories of social relations as social exchanges. The central feature of both theories is the role playedby the notion of values. Also important is how other components of social relations (influence,norms, rights, obligations, authority, social equilibrium, etc.) are connected in terms of valuesexchanged between individuals.? Work partially supported by CNPq and FAPERGS.


19

Piaget [7, 6] follows his well established logical operatory approach to the modelling of humanaction, and defines an algebra of exchange values whose equations are able to characterize situationswhere equilibrium and disequilibrium of social exchanges are manifest. Homans [2, 3] follows thismethodological habit of building on the experimental results obtained by others to fit them into asingle, unified deductive framework, strongly biased by his orientation to Behavioral Psychology.

Both theories are marked by an abstraction effort: both authors abstract away the origins of thevalues involved in social exchanges, that is, the theories are both not concerned with the reasonsand ways the individuals assign values to their own actions, or the actions performed by others.The theories are only concerned with what roles values play in social interactions, that is, in howindividuals take values into account when they interact. Curiously enough, both authors were verydeep readers of the early 1900’s economist Vilfredo Pareto, and both acknowledge their debt toPareto in their choice of an exchange-based explanation of social interactions.

On the other hand, the directions the authors follow in the development of the theories are notthe same. Piaget, being a psychologist himself, was able to fully abstract the strictly psychologicalaspects of the problem, and comes out with a purely sociological model, where the notion of socialequilibrium is the most central one. Homans, being a sociologist himself, was more interested in themotivational side of social interactions, and heavily used B. Skinner’s theory of human behaviorto try to model the motivational aspects of the individuals during the interactions.

Essentially, one can say that Piaget’s theory establishes a descriptive model of social exchanges,using to notion of values and the algebraic operations defined on them to characterize exchangesfrom the point of view of their equilibrium or disequilibrium. Homans’ theory, on the other hand,establishes a particular “causal” model of social exchanges.

Also, an important feature marks a difference between their respective objects of study: whilePiaget is more interested in the analysis of single performances of elementary exchanges, composedof just one stage of type I followed by one stage of type II (see below), Homans is more interestedin the continued performances of such elementary exchanges.

We consider that Piaget’s theory is more fundamental then the one by Homans, in the sensethat Homans results seems to be better explained and understood when related to the overallframework of social exchanges built by Piaget. So, we start we Piaget.

Piaget’s theory of social exchanges. Piaget [7] defines a social exchange as an exchangeof services between to individuals, where a service is any action performed by an individual thatcauses a reaction in the other one, and that can be assigned some value by each of them.

We illustrate with the problem at hand the kind of situation Piaget models in his theory: ina collaborative learning environment, two students are doing some researches, each one about adifferent topic. One of the students may send to a classmate a document with important informationfor the research of that classmate. That document represents a service provided by the first studentto his classmate, and has an associated cost (e.g., the time needed to find the document). To theclassmate, that document represents a benefit, because it allows him to go deeper into his research.Its only natural for the classmate to feel himself in debt with the colleague and to feel obliged topay back (in the same way, or in another way) when opportunity comes.

Social exchanges are assumed to occur in stages, which can be of two types. Let α and β betwo individuals. The two types of stages of social exchanges between them are as shown in thetemporal charts of Figure 1.

In stages of type Iαβ , individual α performs a service to individual β and obtains a credit forthat. In stages of type Iβα, individual α charges individual β for that credit, requiring β to performsome service on behalf of α. The variables in Figure 1 represent the different kinds of exchangevalues involved in an exchange. Clearly, the roles of α and β can be exchanged, in the performanceof the services. An exchange may involve any number of stages of types Iαβ , Iβα, IIαβ and IIβα.


20

a b

r ab

s ba

t ba

vab

vab

t ba

r ba

s ab

a bI ab II ab Values:

1) rαβ is the cost of the service that α per-formed to β;2) sβα is the satisfaction that β got fromthat service;3) tβα is the debt that β acknowledges toα, due to the reception of the service;4) vαβ is the credit that α acquired, due tothe debt acknowledgement of β;5) vαβ is the credit that α charges on β;6) tβα is the debt that β acknowledges topay to α;7) rβα is the service that β pays back to α;8) sαβ is the satisfaction that α receivesfrom the service paid back by β.

Fig. 1. The two types of stages of social exchanges and the values involved in them.

At any time, in an exchange, it is possible to determine a balance of exchange values, given byB = (rαβ , rβα, sαβ , sβα | tαβ , tβα, vαβ , vβα).

To characterize the equilibrium of social exchange, three conditions of equilibrium are definedon the values in the final balance of the exchange. Two conditions refer to the internal equilibriumof the different types of stages, while the third referes to the equilibrium between the differenttypes of stages:

Iαβ : (rαβ = sαβ) ∧ (sαβ = tαβ) ⇔ (vαβ = rαβ)IIαβ : (vαβ = tβα) ∧ (tβα = rβα) ⇔ (sαβ = vαβ)

IαβIIαβ : rαβ = sαβ

The first condition refers to α receiving full credit for the service he performed. The secondcondition, to α receiving full satisfaction from the credit he is charging β. The third conditionreferes to α finally receiving full satisfaction in return for the service he initially performed to β.

Then, by definition, an exchange is said to be in equilibrium if its balance of exchange valuessatisfies all such conditions. Conditions of disequilibrium may arise if, for instance, β’s satisfactionwith the service performed by α is less than the cost of such service, or if β acknowledges a debtwhich is less than his real satisfaction with the service.

Given that credits and debts may be forgotten – purposely or not – by the individuals, and lostas time passes. Piaget’s problem is then to explain how such values are conserved along time, as ithappens when somebody pays his debt long time after receiving the service that originated it. Hisexplanation is that is the very purpose of social rules and norms: many of them are establishedprecisely to guarantee that values will be conserved, and debts will be payed.

Homans’ theory of social exchange. Homans steps to a theory of social exchanges [2, 3]follow, in a general way, the same steps of Piaget: abstraction of the way the individual assignvalues to each other services; consideration of stable situations, where individual behaviors andvalues don’t change, etc. On the other hand, Homans’ goal is to figure out an explanation forsocial behaviors, and behavioral psychology enters his theory as a main tool.

Given that two individuals are continuously interacting in a given way, Homans’ problem is todetermine the reasons why they keep interacting that way, and he seeks an answer connected tothe values exchanged between them.


21

From a behavioristic point of view, if α keeps performing a service s to β, while β keepsperforming a service s′ to α, the only possible reason for the continuation of such social behavioris that the value of β’s service acts as a reinforcer to α’s behavior, and vice-versa. More precisely,Homans calls profit such reinforcement values, and defines it as profit = reward − cost so that thereinforcement values are profitα = sαβ − rαβ and profitβ = sβα − rβα. Profit seen as reinforcersare considered to be the reasons for social behaviors. Exchange values exist in order to reinforcersfor social behaviors exist.

Clearly, from Homans’ point of view, reinforcement occurs only when, in stages Iαβ and Iβα,it happens that vαβ > rαβ and vβα > sβα, respectively, or in stages IIαβ and IIβα it happensthat sαβ > vβα and sβα > vβα, respectively. That is, reinforcement occurs only when, fromPiaget’s point of view, the whole exchange has a positive balance. Consequently, Homans’ notionof equilibrium is different from Piaget’s: for Homans, a social interaction is in equilibrium if itkeeps going on, with the individuals reinforcing each other’s behavior continuously.

Given that, as known from Game Theory, interactions where individuals seek strictly theirmaximal profits may lead to maximal losses, Homans’ problem becomes thus to explain how socialequilibrium is possible and equilibrated situations happen at all. His solution is that individualsmay also take distributive justice into account, so that – instead of maximal profit – they may seeksituations where everyone is fully satisfied with his profit, even though it is not maximal. That is,Homans defines equilibrium as what is now known as a Pareto equilibrium, ending up by assigningto social norms the same estabilizing role that Piaget assigns to them.

3 Sample Computational Applications of Exchange Values

We have been trying to explore ways of modelling social interactions as exchanges of values inthe context of computational systems. In [8], we have presented a way to explore exchange valuesas a tool for the social reasoning of intelligent agents of multi-agent systems. In [9], we suggestedthat exchange values may be a valuable tool for multi-agent based social simulation. The concreteapplication we envisioned in both works was that of modelling agents involved in political elections,dealing with values concerned with votes, political promises, lobby actions, and the like.

In [1], we introduced an equilibrium supervisor, that is, an agent that monitors the valueexchanges between two interacting agents and that, when required, is capable of recommendingsome value exchange between the two interacting agents in order to bring them back to equilibrium.The supervisor models the value exchange process as a Markov decision process with a terminalstate (the equilibrium state), with the set of exchange values represented by a set of numericintervals defined over a range of numerical values. At each non-equilibrium state, the adequaterecommendations for reequilibration follow from the (stationary) optimal policy of the MDP.

4 Modelling of Collaborative Learning Interaction

In this section, we build on our previous experience with the use of exchange values in the modellingof social interactions in computational contexts to present tentative solutions to the problem ofthe role exchange values may play in the modelling of interactions in the context of collaborativelearning environments.

Of course, the first question to meet is if modelling interactions in educational settings amountto the same as modelling interactions in other (e.g., industrial work) settings. Considering theprinciples adopted by both Piaget and Homans, we would expect the following: from the point ofview of the contents of the exchanges, a clear difference should appear, since the concepts, actionsand objects involved in such different kinds of interactions are clearly different. However, from thepoint of view of the form of the exchanges, and of the way exchange values are manipulated, one


22

would expect no observable difference, because – precisely – both theories are proposed as theoriesof the general way exchange values are handled in human interaction.

Given that, we think that exchange values may play a few different modelling roles in the contextof collaborative learning environments: 1) they may represent the values interacting learners assignto each other’s contributions to the collaborative learning process; 2) they may represent the valuesusers assign to the services the environment itself provides to the users.

In the first case, values may be helpful as a complementary tool in the modelling of the wholelearning process, as they capture in certain way important aspects of the dynamics of the inter-actions between the learners. For instance, equilibrium of exchange values between two learnersindicates that they are being mutually beneficial in their interaction. In the second case, values maybe helpful as feedback information about the quality of the services provided by the environment.

In the following, we concentrate on the first of the two above mentioned ways of applyingexchange values in the modelling of collaborative learning interaction – namely, to assess eachother’s contribution – in order to give some tentative answers to the Workshop’s questions.

1) Representation and Design Considerations. As mentioned in section 2, a distinguishedfeature of exchange values is that they have a qualitative nature, since they are not supposed tobe economic values. This means that the representation of values is, in principle, non numeric.

There are many alternatives for the computational representation of qualitative data. The areaof Qualitative Reasoning, in AI, developed several such representations. Fuzzy logic has its ownapproach. In our work [1], we used numerical intervals for such purpose.

Given that exchange values are acquired in some way (see below), their integration in a unifiedmodelling of collaborative learning interaction may represent an important contribution to a widerpicture of the interaction, for exchange values summarize in a way the subjective evaluation anindividual is making of the interaction in which he is involved.

2) Implementation Considerations. Implementation considerations mix themselves withdesign considerations. The main implementation problem is that of the acquisition of exchangevalues: how is the system to get the evaluations that learners make of each others contribution tothe collaborative learning process? The simplest alternative is to require learners to declare suchvalues. This is an alternative very commonly chosen on the web, were a site requires its users toevaluate the documents the site makes available to them.

An automated alternative is making the environment deduce the exchange values from externalcues provided by the learners. For instance, a learner showing a high rate of reuse of documentprovided by another learner may indicate that the document has a high value for the first learner.Also, analysis of dialogues may reveal the values the learners attach to each other’s contributions.

Granularity of the representation of general social values is an interesting topic. Exchange valueswere proposed by Piaget and Homans as the most elementary kind of social values, namely, valuesthat are assigned to concrete, well defined actions performed by individuals.

Taking exchange values as a foundation, higher level values may be incorporated into thepicture: reputation can be defined as a result of a history of exchange values; moral values can bedefined as values that support rules designed to keep interactions in equilibrium; economic valuescan be defined as a special kind of exchange values, of a quantitative kind; juridic values can bedefined as values that support laws that regulate rights and duties concerning services.

In principle, the automated handling of exchange values should not represent a high computa-tional cost in a learning environment, mainly because it seems exchange values need not be usedon a real time basis. They may be handled by a background process and brought to foregroundonly in special occasions, when they are already summarized in stable overall balances.

3) Evaluation Considerations. Accuracy of the representation of exchange values seems tobe the main difficult problem. Except for the case where learners explicitly declare their evaluations(and can be trusted on that), any automated way of acquiring the exchange values involved in an


23

interaction is subject to strong criticism. But, in this respect, modelling exchange values seemsto be in no way different then modelling any other aspect of a learner. Teachers and learners canprovide feedback that help correct an automated modelling of exchange values. Naturally, suchfeedbacks are useful just to the extent that they are reliable.

Repositories of histories of exchanges and their associated exchange values may provide, throughdata mining or some other technique, useful learners exchange profiles, that may help environmentsto support learning interactions in a more adequate way, for instance, by allowing them to classifylearners according to their preferred ways of participating in learning interactions, or by anticipat-ing learners preferences for one kind or another of services provided by others.

5 Conclusion

Reasoning about frequencies of interactions between learners in collaborative learning environ-ments, without qualifying such interactions with respect to the value they have for the learners,seems to us to be an insufficient basis for solving the problem of automating the support for learnersin those environments. Exchange values can help with such problem.

Exchange values capture the qualitative economy of social exchanges and, thus, may give apicture of the collaborative learning interaction that purely operational or even strictly cognitivemodelling cannot give. Exchange values summarize in themselves all the subjective (affective,moral, motivational, etc.) aspects of an interaction and can thus be used as a sound basis for richmodels of such aspects of collaborative learning interaction.

We have used in the paper the basic form of the theory of exchange values, as it was initiallydeveloped by Jean Piaget and George Homans. Piaget has been much less influential than Homansin the subsequent development of the theory. For instance, Homans [5], even in 1987, doesn’t citesPiaget. But, in our understanding, Piaget’s account of exchange values is more fundamental thanHomans’ account, mainly because of the (qualitative) algebraic formalization he provides.

We haven’t had yet the opportunity to run into more recent developments of the theory ofexchange values (see, for instance, the works cited in [5], and their subsequent developments). Butthat is exactly the future work we will be engaged in.

References

1. Dimuro, G. P. and Costa, A. C. R. Interval-based Markov Decision Processes for Planning Interac-tions Between Two Agents. PARA’04 – Workshop on the State-of-the-art in Scientific Computing.Copenhagen, June 23-24, 2004.

2. Homans, G. C. Social Behavior as Exchanges. Amer. J. Sociology, vol. 63, 1958. p.59–606. Also in [4].3. Homans, G. C. Social Behavior – Its Elementary Forms. harcout, Brace & World, New York, 19614. Homans, G. C. Sentiments adn Activities. Free Press, New York, 1962.5. Homans, G. C. Behaviorism and Post-Behaviorism. In Giddens, A. and Turner, J. (eds.): Social Theory

Today. Polity Press, 1987.6. Piaget, J. Sociological Studies. Routledge, London, 1988.7. Piaget, J. Essay on the Theory of Qualitative Values in Static (’Synchronic’) Sociology. Etudes

Economiques e Sociales. Georg, Geneve, 1941. p. 100–142. English translation in [6].8. Rodrigues, M.R.; Costa, A.C.R. and Bordini, R. A System of Exchange Values to Support Social

Interactions in Artificial Societes. In: Proc. of the Second Int. Conf. on Autonomous Agents andMultiagents Systems, AAMAS 2003, Melbourne, Australia, 2003. ACM, New York, pp. 81–88.

9. Rodrigues, M.R. and Costa, A.C.R. Using Qualitative Exchange Values to Improve the Modelling ofSocial Interactions. In D. Hales, B. Edmonds, E. Norling, and J. Rouchier (eds.), Proc. 4th Workshopon Agent Based Simulations, Melbourne, Australia, 2003. Springer, Berlin, LNCS vol. 2927, pp. 57–72.


24

Using Machine Learning to Predict Trouble During Collaborative Learning

BRAD GOODMAN, FRANK LINTON, GUIDO ZARRELLA, and ROBERT GAIMARI

The MITRE Corporation, 202 Burlington Road, M/S K302, Bedford, MA 01730 USA, email: {bgoodman, linton, jzarrella, rgaimari}@mitre.org

Abstract. Our goal is to build and evaluate a web-based, collaborative distance-learning system that will allow groups of students to interact with each other remotely and with an intelligent electronic agent that will aid them in their learning. The agent will follow the discussion and interact with the participants when it detects learning trouble. In order to recognize problems in the dialogue, we investigated conversational elements that can be utilized as predictors for effective and ineffective interaction between human students. In this paper we discuss our representation of participant dialogue and the statistical models we are using to determine the effectiveness of group interaction. Key words: Collaborative learning, intelligent agent, dialogue modeling, student modeling

Introduction

Our research project is exploring techniques for providing stimulating peer learning experiences in web-based, collaborative distance-learning environments. We are leveraging intelligent tutoring system and collaborative learning technologies (Lesgold et al. 1992; Lund et al. 1996) to develop an artificial learning agent – a learning companion (Chan and Baskin 1988) - to collaborate with human peers. The agent will follow the discussion and interact with the participants when it detects learning trouble of some sort, such as confusion about the problem they are working on or a participant who is dominating the discussion or not interacting with the other participants. In order to recognize problems in the dialogue, we first examined the dialogue act underlying an utterance and the role that a participant is playing as the dialogue progresses. The goal is for the simulated peer to play supportive roles based on the instructional needs of the human learners. This paper describes the results of our pursuit of indicators for effective and ineffective collaborative learning. We build on a collaborative-learning infrastructure developed in our previous MITRE research project to bridge distance and time barriers between on-line learners.

Our prior research

We have studied human interaction in collaborative distance-learning situations to provide the foundation for a peer learning agent. A key research question investigated was determining when and how a learning agent should intervene. Such intervention first requires the recognition of when the learning process is in trouble. Table 1 illustrates such a situation. John’s inquiry is ignored completely by Mark and Mary; Mary’s comment is disregarded by Mark. The recognition of these kinds of problems must be done as the dialogue progresses so that the intelligent peer agent can intervene in a timely manner.


25

Table 1. Example learning problem

Mark: We need to add a discriminator. John: What’s that? Mary: No we don’t. Mark: I’ll add one.

We studied the dynamics of collaborative learning groups by observing students working together to solve a common problem in software design using the Object Modeling Technique (OMT) methodology (Rumbaugh et al. 1991). Our initial study took place in a classroom where students worked in small groups (Goodman et al. 1996). We followed our initial study with the development of a collaborative learning environment for OMT. Our goal was to provide a virtual environment that would permit us to analyze the peer-to-peer dialogue and tool actions we needed to identify the strengths and weaknesses of a group’s interaction. The collaborative environment employed a sentence opener-based chat interface (McManus & Aiken 1995) and a shared OMT workspace tool. The sentence-opener interface allowed the dialogue act (also known as communicative or speech acts) underlying each student utterance to be logged; the shared OMT workspace tool permitted student tool actions to be recorded. We ran an experiment in which groups of three subjects used our collaborative environment to solve a software design problem with OMT.

Soller and Lesgold (2000) extend our initial research. They point out that supporting group learning requires understanding the process of collaborative learning. This understanding can be achieved with a fine-grained sequential analysis of the group activity and conversation. Dialogue acts provided the representation for communication between collaborators. They discuss the merits of applying different computational approaches for modeling collaborative learning activities such as the transfer of new knowledge between collaborators. We adopted their revised version of our CSCL software but focused on a different aspect of collaboration and chose a different modeling technique for our research. Dialogues were collected at the University of Pittsburgh (Soller, 2002) and the MITRE Corporation to provide a corpus for study for both projects.

Modeling Student Dialogue

Using the original collaborative learning environment for a new experiment, we obtained data from fourteen groups of three learners given 90 minutes to solve an object-oriented analysis and design problem. The subjects were paid undergraduates and MITRE employees. We logged the learners’ chat sessions and their use of the drawing and agenda tools. Based on an analysis of these logs, we believe it may be possible to draw inferences from the behavior patterns to build user models and perform instructional interventions. We present the rationale in this section.

Dialogue acts

To get at the underlying thrust of learners’ utterances without using full-scale natural-language understanding, we have elected to use a chat tool with sentence openers. Sentence openers are phrases that comprise the first few words of a sentence (McManus & Aiken 1995). They provide a natural way for users to identify the intention of their conversational contribution without fully understanding the significance of the underlying dialogue acts (Searle 1996). To enter an utterance in the chat tool, learners must first select the most suitable sentence opener; they may then input the remainder of the utterance in their own words. Typical sentence openers are “Do you think” (a


26

Request act), “I think” (an Inform act), “Ok. Let’s move on” (a Task act), and “Okay” (an Acknowledgement act). The set of dialogue acts and their corresponding sentence openers that we use were determined empirically in earlier work reported elsewhere (Soller et al. 1998). We collected dialogue acts using the sentence-opener chat interface and found they provided a solid foundation for modeling the collaborative discussion. Dialogue acts have been shown by others as a valuable way to model student dialogue during collaborative learning (Katz et al. 1999, 2000; McManus & Aiken 1995).

Speaker roles

Speakers in collaborative learning contexts not only convey information to other group members, they often assume group roles to help facilitate the learning and problem solving tasks. We observed students in the collaborative learning experiment stepping in and out of particular instructional roles that helped the group move forward towards solving the problem. These roles, such as Questioner or Facilitator, fit in well with research in small group dynamics (Benne and Sheats 1948).

Our goal in this part of the research was to conduct an experiment to reveal the ways participants in a collaborative learning task interact and the factors that govern those interactions. We wanted to see if (1) the instructional roles played by members of the group could be deduced from machine-inferable factors about the collaboration and (2) whether the presence or absence of particular instructional roles indicated the effectiveness of the learning. Our hypothesis is that the presence or absence of particular roles is a powerful indicator of the status of the on-going learning process. We need to identify problem-solving roles portrayed by the group members during a collaborative session to see if they might indicate progress or lack thereof towards successfully completing an exercise. This undertaking entails examining the relationship between dialogue acts and problem-solving roles. We chose the set of roles developed by Benne and Sheats (1948), and modified them slightly to suit the type of dialogues the system elicits, i.e., dialogues involving the use of a whiteboard and a problem-solving task. Example participant roles include Initiator, Evaluator, Information-Giver, and Information-Seeker. Using a Classification and Regression Tree (CART) model, a decision tree variant, we were able to recognize the roles underlying students’ utterances during an evolving student discussion. We can contrast situations where certain roles arise to those where they do not. When a role is expected but missing, it might indicate a place where intervention is warranted to fill the missing role to facilitate better learning and problem solving.

Predicting Effectiveness and Determining Intervention: An Evaluation of Our Indicators

This section presents an evaluation of the use of dialogue acts and participant roles as predictors of group and individual effectiveness and the need for intervention by an intelligent peer agent.

Determining intervention with dialogue acts, roles, and Hidden Markov Models

The classifier we developed predicts group interaction problems by examining entire segments of a dialogue and searching for sequences that indicate poor collaboration. The classifier was trained on instances of poor collaboration found in the training data taken from our fourteen group sessions described above. Each dialogue was divided into segments defined by changes or shifts in tasks (indicated by a “task” dialogue act). Dialogue segments varied from approximately 5 to 200 utterances in length.


27

Each dialogue segment was tagged as an instance of “good” or “poor” collaboration by one of the experimenters. Good collaboration occurs when subjects share information, help each other, offer opinions, acknowledge and respond to one another’s contributions, and don’t dominate a discussion. Progress towards the solution to the students’ assigned exercise is usually present. Poor collaboration results when one or more of the good collaboration traits are missing. It is usually seen with a lack of progress towards the solution but that isn’t necessarily the case. A dominant participant, for example, might indeed move the group towards a solution but at the cost of group cohesiveness and a potential for leaving a group member behind.

Fourteen experimental protocols were coded. We divided the good and poor collaboration examples into training and test sets. We chose Hidden Markov Models (HMM) (Rabiner 1989) as the machine learning technique, following the approach of Soller and Lesgold (2000). However, instead of coding knowledge sharing segments, we coded segments of good/poor collaboration. We trained one HMM to recognize the good collaboration and another one for poor collaboration dialogue segments. The output of the HMM is the probability that a dialogue segment should be classified under a particular category. The input to our first HMMs is the sequence of dialogue acts.

While the HMM recognized 10 out of 11 good dialogue segments correctly, it only recognized 1 out of 3 poor dialogue segments correctly. Next, we attempted a similar experiment in which we used the user roles as inputs in place of the dialogue acts. The resulting HMMs were trained on the same datasets as above and were tested to measure how well they could detect good and poor collaboration. In these results, the HMM performance improved slightly, recognizing 11 out of 11 good dialogue segments but only 1 out of 3 poor segments correctly. In both experiments, the HMM classifiers failed to effectively separate poor and good collaboration segments.

Soller and Lesgold (2000) also utilized dialogue acts to build HMMs to recognize collaborative learning issues. In their case, they attempted to recognize effective and ineffective knowledge sharing between group members (Soller 2002). Our dialogue segments used in training HMMs were often substantially shorter or longer than their segments (we allowed exchanges as few as five utterances and had no upper limit on length). Our segmentation may have led to poorer results. However, we had to work within these tight constraints since a simulated peer cannot help the group if it does not respond soon after a problematic dialogue sequence - detecting trouble long after it occurred would only lead to more trouble! Soller and Lesgold were searching for problematic knowledge sharing episodes after the collaborative learning session had ended.

We felt our results were inconclusive. The good examples dominate; there were too few poor collaboration examples available for training. These outcomes led us to conclude that we should emphasize other conversational elements as well in our analysis. We also believed that good and poor collaboration had to be broken down into more specific situations for machine learning techniques to work effectively. These results led us to examine the use of neural networks as described in the next section.

Determining intervention with dialogue features and neural networks

Given the failure of HMMs to significantly detect globally poor collaboration sequences with the amount of training data available, we decided to (1) examine smaller and more localized sequences of poor collaboration and (2) utilize neural networks which are better suited to the type of data available to us. We built a number of different networks, each responsible for intelligently combining features of the dialogue to evaluate a single aspect of collaboration. We also expanded the set of dialogue features, providing the networks with information about the surface features of the utterances in addition to data on the sequence of dialogue acts. These extra features included the length of the


28

utterances, the pace of the utterances, certain keywords such as “but” or “?”, word overlap between the current and previous utterances, and the identity of the speaker.

Rather than attempt to detect overall poor collaboration in a long sequence of events, the neural networks were used to examine very small segments of dialogue for indicators of poor collaboration. For example, good collaboration requires that participants respond in a timely fashion to the questions and suggestions of others. Students whose questions or suggestions are not acknowledged can become discouraged and disenfranchised.

Training a system to analyze this aspect of group behavior presents two challenges. First, can we detect when a student makes an utterance that must be acknowledged? Second, can we determine when the group does not respond with such an acknowledgement? We coded three dialogues with markers for these adjacency pairs. We then built two neural networks, one for detecting a question and one for detecting the responses.

The first network is a two-layer back-propagation network with 7 inputs, 5 hidden units, and 1 boolean output. It was trained on 218 segments from one dialogue and tested on 77 segments from other dialogues. We achieved 96% accuracy on this small test set with a kappa of .884. The network scored 85% using only dialogue acts as input, and 87% using only surface features. The 96% mark was reached using dialogue acts and the surface features of utterance length and the presence of a question mark.

The corresponding network was trained to recognize responses to utterances in need of acknowledgement. It is a two-layer back-propagation network with 6 inputs, 5 hidden units, and 1 boolean output. The neural network tests each utterance that occurs within two minutes of a previously detected question. This network was trained on 665 segments from one dialogue and tested on 234 from other dialogues. It scored 86% accuracy on the test set with kappa of .50. The 86% accuracy was reached using contextual dialogue acts, timing, speaker, and length as inputs.

An advantage of this approach is that it allows us to break each dialogue into hundreds of small segments containing one opportunity for good or poor collaboration. Increasing the number of training examples we could extract from our limited data was a critical element to the success of this methodology. Although this approach does not make a determination of the overall health of the group collaboration effort, it can be used to build a user and group model containing information about specific aspects of collaboration. This model can be analyzed in turn to estimate the types of problems occurring in the group and to suggest possible areas requiring agent intervention.

Discussion

The results of our HMM and neural network tests demonstrate that important characteristics of group activity during collaborative distance-learning can be detected using machine learning techniques. While we could not match Soller and Lesgold’s success with detecting knowledge sharing episodes using HMMs, we did achieve success using neural networks. With a larger data pool, we may be able to construct HMMs that effectively classify good and poor collaborative learning.

The presence of a chat tool permits learner-learner interaction that is not formalized and therefore is beyond the capability of the student modeling module to directly interpret. These results are preliminary and any conclusions must be tentative, however it appears that even without high-quality natural language understanding, dialogue acts


29

and simple surface features provide information that allows the student modeling module to make useful inferences. These inferences contribute to a determination of any actions the peer agent should take to benefit the learners.

Future Directions

As web-based collaborative interactions grow and become a common form of learning in society, it will become crucial to increase their communicative effectiveness. Ubiquitous interface technologies and higher bandwidth connections will increase the richness of the communication among students. Spoken communication, for example, will replace text-based chat among participants. We have begun exploring the contributions prosodic features (Forbes-Riley & Litman, 2004) can provide as indicators of learning difficulty. In addition, we are moving away from sentence-opener interfaces to ones that use automated dialogue act recognition.

Acknowledgements. The MITRE Technology Program supported the research described here. We collaborated on this research with Amy Soller at the Learning Research and Development Center (LRDC), University of Pittsburgh. Dr. Soller worked on our earlier collaborative learning project and expanded one aspect of that research for her doctoral dissertation on knowledge transfer in collaborative learning. We ran joint experiments and shared the data.

References Benne, K. & Sheats, P. (1948). Functional Roles of Group Members. Journal of Social Issues, 4:41-49. Chan, T. & Baskin, A. (1988). Studying with the prince. The computer as a learning companion. In Proceedings of

the ITS-88 Conference, (Montreal, Canada), 194-200. Forbes-Riley, K. and Litman, D. J. (2004). Predicting Emotion in Spoken Dialogue from Multiple Knowledge

Sources. Proceedings of HLT-NAACL 2004, Boston, 201-208. Goodman, B., Soller, A., Linton, F., and Gaimari, R. (1996). [Videotaped study: 3 groups of 4-5 students each

solving software system design problems using Object Modeling Technique during a one week course at The MITRE Institute]. Unpublished raw data.

Katz, S., Aronis, J., & Creitz, C. (1999) Modeling Pedagogical Interactions with Machine Learning. In S. P.l Lajoie and M. Vivet (Eds.), Artificial Intelligence in Education. Amsterdam: IOS Press, 543-550.

Katz, S., O’Donnell, G., and Kay, H. (2000). An Approach to Analyzing the Role and Structure of Reflective Dialogue. In International Journal of Artificial Intelligence in Education, 11, 320-343.

Lesgold, A., Katz, S., Greenberg, L., Hughes, E., & Eggan, G. (1992). Extensions of intelligent tutoring paradigms to support collaborative learning. In S. Dijkstra, H. Krammer, J. van Merrienboer (Eds.), Instructional Models in Computer-Based Learning Environments. Berlin: Springer-Verlag, 291-311.

Linton, F., Goodman, B., Gaimari, R., Zarrella, J., and Ross, H. (2003). Student Modeling for an Intelligent Agent in a Collaborative Learning Environment. In Proceedings of the International Conference on User Modeling, Johnstown, PA.

McManus, M, & Aiken, R. (1995). Monitoring computer-based problem solving, Journal of Artificial Intelligence in Education, 6(4), 307-336.

Rabiner, L. R. (1989). A tutorial on hidden Markov models, Proceedings of the IEEE, vol. 77, pp. 257-286. Rumbaugh, J., Blaha, M., Premerlani, W., Eddy, F., and Lorensen, W. (1991) Object-Oriented modeling and design.

Englewood Cliffs, NJ: Prentice Hall. Soller, A., Goodman, B., Linton, F., and Gaimari, R. (1998). Promoting effective peer interaction in an intelligent

collaborative learning environment. Proceedings of the Fourth International Conference on Intelligent Tutoring Systems (ITS 98), San Antonio, TX, 186-195.

Soller, A., & Lesgold, A. (2000). Modeling the Process of Collaborative Learning. International Workshop on New Technologies in Collaborative Learning, Awaji-Yumebutai, Japan.

Soller, A. (2002). Computational Analysis of Knowledge Sharing in Collaborative Distance Learning Doctoral Dissertation. University of Pittsburgh.


30

A model and a pattern for the collection of collaborative action in CSCL systems

Alejandra Martínez1, Luis A. Guerrero2, César A. Collazos3

1 Dept. of Computer Science, Universidad de Valladolid, Valladolid, Spain [email protected]

2 Dept. of Computer Science, Universidad de Chile, Santiago, Chile [email protected]

3 Dept. of Systems, Universidad del Cauca, Popayán, Colombia [email protected]

Abstract

There is an increasing interest in the CSCL field towards the definition of frameworks that support analysis of interactions in order to understand or to regulate collaboration. These analysis processes draw on the automatic collection of data about the collaborative processes for its further analysis by manual or automatic means. Despite this interest, current proposals solve this automatic collection using ad-hoc solutions, and thus they do not consider how this problem can be solved in a modular and reusable manner, so that it could be applied to different collaborative situations and analytical approaches. This paper shows how CSCL can benefit from the field of software engineering by the adaptation of the command design software pattern to the problem of CSCL data collection. In order to perform this adaptation, we draw on a model of interaction that defines the concept of collaborative action as the basis of any interaction, which is also described in this paper. These two aspects: the concept of collaborative action and the adaptation of the command pattern to the problem of CSCL data collection will allow us to address conceptual as well as implementation issues related to the modelling of interactions in CSCL.

1 Introduction Analysis of interactions has become a main research topic in the last years in CSCL. However, research has mainly focused on conceptual issues [7] or on the development of experimental prototypes focused on testing a particular collaboration support model [6]. While these approaches have shown the interest of this CSCL work line, the area has so far neglected the problem of the design of these systems from a software-engineering point of view. We claim that this is an important topic, as we need to provide the developers with conceptual and practical tools that facilitate the desired integration of analysis functions in CSCL systems. This integration has to meet software quality criteria, such as modularity and reusability. We propose in this paper to borrow from the software design patterns point of view [4] in order to identify and propose design solutions that meet these objectives.

The collaboration management cycle proposed in [6] defines the phases that these collaboration support tools have to go through in order to perform their tasks. The first of these phases consists on the automatic collection of data about the collaborative processes. We are going to focus on this phase, as it is the one where the interface between the functions oriented to mediate collaboration and to perform the analysis is established. Moreover, being the first phase, it is the one on which the rest of the analysis processes rely.

In order to apply a software pattern to the collection of data in CSCL, some steps are needed in advance. We need to provide an interaction concept that is both appropriate for the domain as well as operationally representable by a computer. We propose the concept of collaborative action to fulfill this objective. Once this concept is defined and described, we propose how a well-known software design pattern - the command pattern [4],- can be used to implement the collection of data in systems that follow the collaboration management cycle.


31

This work is part of ongoing research performed by the authors that aims at the definition of a component- based framework for the definition of reusable, modular and configurable solutions for the implementation of collaboration support systems [9]. It also draws on previous authors’ experience in the proposal of a pattern-based system for CSCW applications [5].

The rest of this paper is structured as follows: section 2 presents the concept of collaborative action, and section 3 presents its computational representation using an UML diagram. Then, section 4 introduces the command design pattern and how a logging tool in a CSCL context can use it. Finally, section 5 presents the main conclusions and further work.

2 Collaborative action concept The concept of collaborative action is not easy to define. Although it has been extensively used in the literature, either its meaning has been taken for granted or it has been defined specifically within the context of each approach. The Merriam-Webster's Collegiate Dictionary defines interaction as “mutual or reciprocal action or influence”. Therefore, interactions can be conceptualised as actions that comply with a specific characteristic: reciprocity. Then, we are to answer what means reciprocity in our CSCL context and how it can be detected. In dialogue-based analysis, reciprocity seems to be an easy issue, if we assume that any utterance is an interaction. However, several authors have shown that this is a rather simplistic view. [1] states that the “degree of interactivity between pairs is not defined by the frequency of interactions, but for the degree in which they have an influence in the cognitive processes of participants”, which gives an idea of the complexity of the concept. However, this statement can be challenged for two reasons. On the one hand, it is difficult to apply operationally. On the other hand, it focuses on the cognitive view of interactions, leaving apart the participatory aspects, which we must consider when we want to study interactions in all their dimensions. Another well-known challenge to the approaches that rely on explicit interactions is the presence of silence in many situations of real collaboration. Indeed, silence can be almost as significant, if not more, than explicit utterances [7], and therefore we should not rely exclusively on explicit discourse, but include actions when performing analysis.

In conclusion, we need a definition of interaction able to deal with actions and discourse, covering cognitive and participatory aspects of interaction, simple to process and able to deal with silence and inactivity. Taking this into account, we propose the following definition for interaction as “an action that affects or can affect the collaborative process. The main requirement for an action to be considered a possible interaction is that the action itself or its effect can be perceived by at least a member of the group distinct of the one that performed the action”. It provides a generic view of interaction, without restricting it to a particular source of data or analytical perspective, and gives an operational criterion to select appropriate input for interaction analysis. It is also able to deal with the aforementioned problem of silence.

Before we continue with the rest of the proposal, we want to point out that the study of human action from a situated standpoint is rather different from the study of behavior in the conductivist paradigm. In order to be fully understood, human action needs to consider the context in which it is taking place [11]. Thus, representation of collaborative action from a situated learning perspective needs to consider context, which is in fact an open question in the modelling and analysis of human action. Next section will elaborate on this issue.

3 Computational representation of collaborative action Once we have presented the concept of collaborative action, we will present our proposal for its computational representation in an open and standard format. As mentioned above, there are several issues one must consider when trying to model interactions. First, we have to face the problems related to the modelling of context in collaborative situations. Next, we have to provide a classification of collaborative action that fits the definition we have presented in the previous section. Section 3.1 will ellaborate on the representation of context, while section 3.2 will present the three main types of action we have identified.


32

3.1 Representation of context

Although research in CSCL currently agrees on the need of considering context when interpreting human action, it is an open issue how this is to be done, and what elements should be considered when modelling it, especially from a computational perspective. Normally, this issue is solved depending on each particular situation, and there are few proposals of a generic representation of context.

A possible exception to this rule is the use of Activity Theory as a framework for the activity representation in its social context [3]. Although we agree that this approach is being successful at broadening the analytical perspective of researchers, we claim that its concepts are rather generic, and need to be complemented with information related to the pedagogical context in which the learners are interacting. As an alternative, we propose to use the concepts of the DELFOS framework for the modelling of context in collaborative learning.

DELFOS was developed specifically for the definition of CSCL situations, taking into account social, pedagogical, and technological issues. It presents a model of collaborative situations, and has been validated by its use in the design of several applications [10]. It proposes the concept of situation to model the general features of a learning environment, including learning objectives, number of expected participants, metaphors, etc. DELFOS provides for the definition of users, roles, objects and groups that intervene in the situations. All these elements are represented in the model, as shown in figure 1.

3.2 Collaborative action representation

The second aspect we face in our proposal is to provide a generic and operational taxonomy for the representation of collaborative action. As mentioned beforehand, existing approaches focus on the representation of a single feature of the interaction, which hinders the desired integration of different sources of data. We aim at integrating dialog and action, as well as automatic and manually collected data in a common structure, by means of a new classification that focuses on the actors that take part in interactions. The main advantage of this approach is that it easily accommodates to the collected data in each system for each type of interaction.

ParticipationIndirectObject 11

Activity

Di rect

Situation

1..*1..*

1..*1..*Role 0..*0..*

Group

0..*0..*

0..*0..*

is composed by

User

Receives

1..*1..*

1

0.. *

1

0.. *

0.. *0.. *

belongs to

Session1..*1..*

Clockdate

Action

1..*1..*

1

0..*

1

0..*

Creates timestamp

Fig. 1. Model of collaborative action represented in UML (Unified Modelling Language). It shows the

classes related with the context as well as the three types of collaborative action that have been defined. The links between classes show the most relevant associations, and the numbers displayed near the classes

show their cardinality for these associations.

The proposal distinguishes between direct, indirect and participation-oriented interactions (see figure 1):

- Direct: They represent the typical idea of an interaction, with a source and one or more receivers. It can be mediated by a channel, and may specify its content, which will normally, but not necessarily, be a dialogue representation.


33

- Indirect: This interaction is characterised by being mediated by an object, and therefore, it is the more common in shared workspace environments.

- Participation: As an action that has no object neither receiver. Represents a generic intervention in a collaborative environments.

Although this model is valid for face-to-face representing actions as well as those that are performed through

the computer, we will restrict our scope to the latter. These automatic actions will always correspond to a command (or set of commands) executed by the CSCL application in response to an event (or to a set of them). Therefore, we have to think on how the commands are to be implemented in the application in order to facilitate the collection of actions. This solution has to be generic, so that it can be adapted to different types of actions and to different CSCL situations. It also has to provide independence between the code of the CSCL application and the code of the collection of data, as they are different functionalities and it is desiderable that one of them can be modified without affecting the other. Next section presents a particular software pattern called the command pattern that meets these requirements, and how it can be applied to solve the logging of actions.

4 Adaptation of the command pattern for the collection of actions The command design pattern is a recommendation for the implementation of all the commands of an application in order to promote independence between the sender of a request and its receiver. The pattern proposes to implement all the commands as objects with a method Execute(), which is the one that fulfills the command functionality. The set of these command objects constitutes the 100% of the functionality of the application, i.e., everything that the application has to do, including any action and the data generated by it.

As shown in figure 2, the key feature of this pattern is an abstract Command class, which declares the interface for executing operations, which in its simplest form includes an Execute() operation. The concrete Command (ConcreteCommand) classes are declared as subclasses of the abstract Command and are in charge of implementing its interface. Each ConcreteCommand class specifies the Receiver of the command by means of an instance variable that stores it. The Receiver can be any class that has the knowledge to fulfill the request. Finally, the Invoker is responsible of calling the Execute() operation in the Command interface.

The interaction between the objects in this pattern works as follows: The Client creates a Concrete Command object and sets its Receiver. The Invoker stores the Concrete Command object that has been instantiated, and issues a request by calling Execute() on the Command object. Then, the ConcreteCommand object invokes operations on its Receiver to carry out the request.

AbstractCommandis_loggeable : Boolean

Execute()

Invoker

ConcreteCommand2is_loggeable : Boolean

Execute()

Client

PerformCommand()

Application (Client)

sets

ConcreteCommand1is_loggeable : Boolean

Execute()

receiver

creates

Logger

writeAction()

sets

logger

Fig. 2. The command pattern adapted to the collection of actions in a collaborative environment


34

This pattern can be easily adapted to support the collection of data by adding a logical is_loggeable instance variable to the abstract Command class. This acts as a switch: if it set to true, the action to which this command corresponds will be logged; if it is false, the action will not be logged for its further processing. The logging processes are performed by the Logger class, that receives the command data and represents it in the desired format using the WriteAction() method. Normally, the output is a text file, but it can take any other structure that is convenient for the subsequent analysis processes. Therefore, as it can be seen in the figure, a Logger class can be considered as a special type of Receiver that is invoked conditionally if the is_loggeable variable is set to true in a particular command.

Once we have explained the pattern and its use for implementing the collection of data in a CSCL system, let us now discuss how to integrate it with the concept of collaborative action that has been proposed beforehand. As it has been mentioned, there is a link between commands and actions. The semantic level of a collaborative action is that of the usual commands one can expect to find in a CSCL application (create an object, issue request, answer to a question, etc.). Sometimes these commands result from the aggregation of a set of lower level or more fine-grained ones. Therefore, it is possible to define the relationship between the concepts of command and collaborative action, either as a one-to-one relationship, or as an aggregation of commands that constitute an action. The command pattern is also appropriate to log an action represented as an aggregation of commands, as it is possible to assemble the low-level commands in a composite command class that represents a particular type of collaborative action to be logged.

In conclusion, we see that the command pattern applies quite straightforwardly to the logging needs of a CSCL application. A positive consequence of this is that we can take advantage from the benefits reported from the use of this pattern. If we recall that the pattern was meant to decouple the object that invokes a command from the one that performs it, CSCL applications that use it will not need to know how to log any of their actions. This will be a responsibility of the concrete action classes. The pattern is flexible, as it allows for the definition of new types of action (through a new specialisation of the abstract class Command). It facilitates also the modification of the logging mechanisms by changing the Logger() class, that will not affect the code of the CSCL application being logged.

Apart from these benefits, the pattern is the base of new functionalities that become very easy to implement with its use. For example, a tool that shows the user (teacher, evaluator, and student) a menu with all the possible actions performed in a particular CSCL tool. The user can choose which of these actions s/he wants to be logged, which will be represented by a true value in the is_loggeable attribute of the class that represents that action. The actions that are not meaningful for a particular analysis and therefore have not been chosen in the menu, are omitted from the log by setting to false this attribute. This eventually results in a more accurate and easy to analyse log than other that simply collected all the data. This is only an example, that illustrates the interest of taking care of software design quality issues in the elaboration of CSCL applications in general, and of analysis functions in particular.

5 Conclusions and future work CSCL needs to take software design issues seriously in order to enhance the quality of the applications in the field. This is not a pure technical problem, as the modelling of actions which is at the core of the proposal needs to take into account issues related to the context, types of interactions, etc. which draw on the needs of the domain. This fact has been reflected in the definition of collaborative action we have presented in the paper.

We have shown with a simple example how a set of conceptual issues have been applied to the computational representation of action. Also how the use of a software pattern can facilitate and enhance the design of a core functionality in CSCL such as the collection of data about collaboration for its further analysis.

The ideas presented in this paper are related with ongoing research of the authors. On the one hand, the model of collaborative action has been used as the base for the definition of a DTD for the computational representation of collaborative action logs in XML [8]. The genericity of the model and the standard character of this language make feasible its use by other CSCL developers, in order to produce data that could be easily shareable between CSCL systems and evaluation tools. This action-oriented approach, as well as the command design pattern are part of a more general architecture for computer-suported collaborative systems that has already been tested with a prototype [9]. Inmediate future work plans include to apply the ideas of the paper with this architecture to other CSCL situations in order to have a better idea of their applicability.


35

Acknowledgements This work has been partially funded by the European Commission Project EAC/61/03/GR009 and the

Spanish Ministry of Science and Technology Project TIC2002-04258-C03-02 as well as by the research mobility program from the University of Valladolid, Spain.

References [1] Dillenbourg, P. (1999). Introduction; What do you mean by “collaborative learning”? In P. Dillenbourg

(Ed.), Collaborative learning. Cognitive and computational approaches. Oxford: Elsevier Science (1999) 1-19.

[2] Dimitracopoulou A. & Petrou A. Advanced collaborative learning systems for young students: Design issues and current trends on new cognitive and metacognitive tools. In THEMES in Education International Journal. (2003)

[3] Fjuk, A., & Ludvigsen, S. The complexity of distributed collaborative learning: Unit of analysis. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.) In Proceedings of European Conference of Computer Support Collaborative Learning (EuroCSCL’01) (2001) 237-243.

[4] Gamma, E., Helm, R., Johnson, R., and Vlissides, J. Design patterns: Elements of reusable object-oriented software, Addison-Wesley, (1995)

[5] Guerrero, L.A. and Fuller D. A Pattern system for the development of collaborative applications. Information and Software Technology, Vol.43, (7) (2001) 457-467

[6] Jermann, P., Soller, A., & Muehlenbrock, M. From mirroring to guiding: A review of the state of the art technology or supporting collaborative learning. In Proceedings of European Conference of Computer Support Collaborative Learning (EuroCSCL´2001) (2001) 324-331,

[7] Littleton, K., & Light, P. (Eds.) Learning with computers: Analysing productive interaction. London: Routeledge (1999)

[8] Martínez, A., Dimitriadis, Y., de la Fuente, P. Towards an XML-based model for the representation of collaborative action. In Proceedings of Computer Support for Collaborative Learning (CSCL 2003) (2003) 379-388

[9] Orozco, P. Asensio, J.I. García, P., Dimitriadis, Y.A., Pairot, C. A decoupled architecture for action-oriented coordination and awareness management in CSCL/W frameworks. In Proceedings of the 10th Int. Workshop on Groupware, (GRIWG’04) (to appear)

[10] Osuna, C., Dimitriadis, Y., & Martínez, A. Using a theoretical framework for the development of educational collaborative applications based on social constructivism. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.), Proceedings of European Conference of Computer Support Collaborative Learning (EuroCSCL´2001) (2001) 577 - 584

[11] Wilson, B., & Myers, K. Situated cognition in theoretical and practical context. In D. Jonassen & S. Land (Eds.), Theoretical foundations of learning environments. Mahwah, N.J.: Lawrence Erlbaum Associates. (2000) 57-88


36

A Computational Model for Differentiating between Action and

Interaction in Shared Workspaces

Martin Mühlenbrock

German Research Center for Artificial Intelligence (DFKI) Saarbrücken, Germany

[email protected] Abstract: Approaches for measuring interaction in shared workspaces mostly take on a simplistic perspective in regarding every action also as an interaction. In contrast, this paper argues that actions in shared workspace differ in their degrees of interaction and defines formal indicators for the automatic evaluation. The approach presented in this paper is based on a hybrid model that uses plan recognition to determine the different dimensions of shared-workspace interaction and Bayesian classifiers for their combination. Keywords: Computational Models of Collaboration, Shared Workspace System, Action-based Analysis, Plan Recognition, Bayesian Reasoning Assessment of Collaborative Activity There is a growing interest in the development of analysis systems for collaborative learning, which are providing the basis for briefing students and teachers about the course of the joint activity as well as computerized approaches for guidance [3]. Recently, a collaborative activity function has been proposed that is based on the number of messages that are posted through the available communication channels including chat and modeling tools among others [2]. However, this collaborative activity function attributes the same degree of interactivity to every action, disregarding the fact that some message might have been posted in isolation. For a shared workspace, this could mean that some activity is regarded has highly interactive when in fact all users have worked completely on their own and their actions have merely been interrelated.

The issue pointed out above is related to the distinction between collaboration and cooperation. Cooperative work is accomplished by the division of labour among participants, as an activity where each person is responsible for a portion of the problem solving, whereas collaboration involves the mutual engagement of participants in a coordinated effort to solve the problem together [5]. That is, collaboration involves a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem.

Dimensions of Interactivity In order to improve the measuring of collaborative activity, for each action its interactive dimensions has to be considered. The combination of these interactive dimensions determines the overall interactivity of the action. Hence instead of simply considering each action that occurs in a shared workspace as an interaction, the context of its occurrence is taken into account. In general, a shared workspace is regarded as a collection of objects and directed edges between these objects. For shared workspaces, the following set of dimensions is proposed to model the interactivity of actions, starting with simple and obvious ones towards more complex ones:


37

• Co-Presence of users (CP): A user’s action has a higher degree of interactivity if other users a present, i.e. have

logged into the shared workspace.

• Synchronicity of actions (SA): The longer the time between to actions, the less interrelated they are. For each action, there is a time interval concerning the preceding action and a time interval concerning the subsequent action.

• Action distance (AD): The more actions are performed between two actions the less they are related.

• Attribute intersection (AI): An action changes one or more attributes of one or more objects. The larger the intersection of affected attributes and objects of two actions the more they are interrelated.

• Object distance (OD): The farer apart two actions are performed the less they are interrelated.

• Object connectedness (OC): The more connected the objects are on which two actions are performed the more elated they are, i.e. actions can be performed on the same objects, on directly connected objects, or on indirectly connected objects.

• Observability of actions (OA): Two actions are more related to each other the better the users can observe not only their own action but also that of the other.

• Visibility of actions (VA): Two actions are more related the more visible their effects are. This depends on the size of the object and the total number of objects that are observable.

• Enabling action (EA): If one actions can only performed because of another actions, they are related

• Impeding action (IA): If one action impedes the performance of another action, they are also related.

• Cancellation of actions (CA): If one action reverses the effect of another action, they are interrelated.

• User Insistence (UI): If one action is performed repeatedly with at least one intervening cancellation, it can be seen as an insisting on the action. All three actions are interrelated.

• Alternation of actions (AA): The higher the degree of alternation between users the higher the interactivity.

These dimensions span a space of interactivity for each action that is performed in a shared workspace. None of these dimensions refer only to one action, but at least two. Interactivity of actions is inherently bound to sequences of actions. The total interactivity is determined by the combination of the different dimension, since the various dimensions contribute to the overall interactivity with different shares. A computational model has been developed to represent as well as detect the interactivity of actions in shared workspaces. It is a hybrid approach based on plan recognition for analyzing sequences of actions and Bayesian classifiers to integrate the different dimensions of interactivity. Analyzing Action Sequences Action-based collaboration analysis tracks user actions and indicates specific patterns of activity and interaction [4]. It is formally grounded on the situation calculus and on approaches for plan and task recognition. The recognition and interpretation of user and group actions is organized hierarchically, starting from basic actions in the shared workspaces, and successively deriving higher-level activity from these. This is achieved by observing user actions in the context of the problem-solving product as well as by relating actions to preceding and potentially subsequent actions. Taking a stream of action messages from the shared workspace as an input, the activity recognition automatically and incrementally infers more abstract concepts related to group activities and interpretations of conflicting and coordinated action sequences.


38

In action-based collaboration analysis, a shared workspace is represented as a collection of objects and directed edges between these objects. Typical basic actions include the creation, modification, and deletion of objects and links among others. Actions are conceptualised by means of operators that specify for each action its pre-conditions, post-conditions, background information, its decomposition into other actions, and a set of applicability conditions called constraints. In this way, each action is formalized by one or more operators. These operators are executable, i.e. they can detect corresponding actions and action sequences in a stream of action messages from a shared workspace. The approach does not rely on domain or task knowledge, though it can possibly use available background information to complement its results. The different dimensions of interactivity as defined above can be recognized by appropriate operators. For instance, the synchronicity of actions (SA) can be determined by means of the following operator:

operator: synchronicity_of_actions(high) decomposition: create_object(Object1, Time1, Actor1) create_object(Object2, Time2, Actor2) background: threshold_short_time(Time) constraints: Time1 – Time2 < Time

Another example is the connectedness of objects (CO). If two actions are performed on the very same object, they are strongly interrelated. If however two actions are performed on objects that are not connected at all, they are not much related to each other. However, there is also middle ground, because objects might be connected to each other by means of explicit links or by means of implicit relations based on spatial arrangements among others. Furthermore, this connectedness can be direct or indirect via connections to intermediate objects.

operator: object_connectedness (high) decomposition: create_object(Object1, Position1, Actor1) modify_object(Object1, Position2, Actor2)

operator: object_connectedness (medium)

decomposition: create_link(Object1, Object2, Actor1) create_link(Objects3, Object2, Actor2)

operator: object_connectedness (low)

precondition: transitive relation(Object2, Object4) decomposition: create_link(Object1, Object2, Actor1) create_link(Objects3, Object4, Actor2)

In these operators, the realization of relations in the workspace include explicit relations given by some link or

edge between objects as well as implicit relations from spatial relations like adjacency. These are some examples of operators, which are the basic building blocks for the activity recognition. A number of operators have been pre-defined in form of an activity library for shared-workspace analysis. A Bayesian Classifier for Interactivity In order to make sense out of the dimensions of interactivity, be it for using it in a system feedback function or further analysis among others, the different dimensions have to be fused to a single interactivity value. The Naive Bayes Classifier technique is based on the so-called Bayesian theorem and is particularly suited when the dimensionality of the inputs is high. Despite its simplicity, Naive Bayes can often outperform more sophisticated classification methods. A Bayesian approach can be used to determine the total interactivity (I) with the maximum a posteriori (MAP) probability in the following way


39

IMAP = arg max P(I | CP, SA, AD, AI, OD, OC, OA, VA, EA, IA, CA, UI, AA)

By applying the Bayesian theorem (the constant denominator can be eliminated because of the argmax), we have

IMAP = arg max P(CP, SA, AD, AI, OD, OC, OA, VA, EA, IA, CA, UI, AA | I) P(I) If we make the simplifying assumption that all dimensions of interactivity are conditionally independent (Naïve Bayesian classifier), we have

∏∈ },,,,,,,,,,,,{

)()( |AAUICAIAEAVAOAOCODAIADSACPS

aPIIMAP = arg max P S

In order to estimate the prior probabilities for the Bayesian learning, the easiest way is to assume an even distribution. Given a sufficient number of training examples that are composed of a set of actions in a shared workspace together with a characterization of their degree of interactivity by the users or someone else, this interactivity classifier can be improved beyond the initial estimation of the interactivity probability as well as adapted to different settings of shared workspace usages. Further Work In this paper we have defined an evaluator of shared workspace activity that can distinguish interactions from actions. It is specified as a hybrid approach based on plan recognition, i.e. action-based collaboration analysis [3], and Bayesian classifiers. The Bayesian classifier for interactivity fusion has been specified, but still needs to be trained with real world examples. For each action the classifier computes a probability for it to be an interaction in addition to being an individual activity. The concept of interactivity as presented here has some interesting properties in common with the notion of focus of attention as used some shared workspace systems. Furthermore, the notion of awareness [1] seems to play some role, the relationship to both needs further research. Acknowledgement

The work presented in this paper is partially supported by European Community under the Information Society Technologies (IST) programme of the 6th FP for RTD - project iClass contract IST-507922.The authors are solely responsible for the content of this paper. It does not represent the opinion of the European Community, and the European Community is not responsible for any use that might be made of data appearing therein. References 1. Dourish, P. & Bellotti, V. (1992). Awareness and coordination in shared workspaces. Proceedings of the 1992 ACM

conference on Computer-supported cooperative work table of contents, pages 107-114. Toronto, Ontario, Canada.

2. Fessakis, G., Petrou, A., Dimitracopoulou, A., (in press) Collaboration Activity Function: An interaction analysis’ tool for Computer Supported Collaborative Learning activities, In 4th IEEE International Conference on Advanced Learning Technologies (ICALT 2004), August 30 - Sept 1, 2004, Joensuu, Finland

3. Jermann, P., Soller, S., & Muehlenbrock, M. (2001). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. In Proceedings of the European conference on Computer Supported Collaborative Learning, EuroCSCL-2001. Maastricht, The Netherlands, March.


40

4. Muehlenbrock, M. (2001). Action-based collaboration analysis for group learning. Amsterdam, The Netherlands: IOS Press, Dissertations in Artificial Intelligence.

5. Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. O'Malley, editor, Computer Supported Collaborative Learning, pages 67-97. Berlin: Springer.


41

Implications of Shared Representations for Computational Modeling

Dan Suthers

University of Hawaii

[email protected]

Abstract: Online collaborators may make use of multiple representational media, including sharedworkspaces that might contain visual reasoning and modeling representations (for example), as well asnatural language communication tools. This paper considers the implications of this fact forcomputational models of collaboration. Examples of collaboration through shared workspaces areexamined to motivate and ground observations about how such computational models will need toboth abstract from yet also be sensitive to the choice of media in which collaboration takes place.

My research is generally concerned with how software tools that support learners’ construction of knowledgerepresentations (e.g., concept maps, evidence maps, evidence tables) are used by collaborating learners, andconsequently how to design such tools to more effectively support collaboration. A previous study (Suthers,Girardeau & Hundhausen 2003) observed that online collaborators treated the graph as a medium through whichcollaboration took place as well as its object, proposing new ideas by entering them directly in the graph beforeengaging in (usually brief) confirmation dialogues in a textual chat tool. In general, actions in the graph appeared tobe an important part of participants’ conversations with each other, and in fact was at times the sole means ofinteraction. These observations led to the questions of whether and in what sense we can say that participants arehaving a conversation through the graph, and whether knowledge building takes place. To answer these questions, Iidentified interactions from our corpus that appeared to constitute collaboration through the nonverbal as well asverbal media, and am engaged in a qualitative analysis of these examples. The purpose of this analysis is tounderstand how participants made use of the structured (graph) representation to mediate meaning making activity,by examining how participants use actions on the representations to build on each others’ ideas.

If collaboration can take place through multiple representational media, computational models of collaboration willneed to either be able to take account of or operate independently of this fact. In this paper I give examples of howparticipants make use of shared representations to collaborate, and consider how models might capture importantaspects of this use. I begin with comments on the scope of computational modeling considered; then describe thestudy from which my data is derived; and conclude with examples and their implications for modeling collaboration.

Computational Modeling of Collaboration with Shared Representations

Computational modeling is undertaken for various reasons, which I will group into generative and recognition.Models that generate a phenomenon – simulations – may be used to test an underlying theory, or applied as a way toautomate a desired behavior. Examples in ITS include various forms of learning companions (Aimeur et al, 1997;Chan 1996). Recognition models take external events as input and translate these into state changes that indicatewhether a certain kind of event has occurred or the input falls in a certain category. In the area of collaborativelearning, recognition models have commonly been applied to identify certain problems and opportunities to which asoftware coach might respond. For example, Constantino, Suthers & Escamilla (2003) show how to identifyimbalances in levels of participation (a problem in collaboration) and how to recognize when a student might haveideas that are different from the group’s difference that might be worth discussing (a learning opportunity). Personalcoaches (one for each collaborating student) use this information to decide whether and how to intervene. Soller &


42

Lesgold (2003) developed a method for detecting effective and ineffective episodes of knowledge sharing,information that could also inform a collaboration coach. Computational models can serve both roles. For example,the Cognitive Tutors (Corebett, Koedinger & Anderson 1997) are at least in theory capable of both generation andrecognition (“model tracing”).

In the present paper, I consider how we might recognize important events in interactions via shared representations.My discussion will fall far short of informing generative models.

What are the primitives of models of collaboration?

Modeling is about structures or patterns: it is not interesting or useful to classify isolated events. A given model willtake events at some level of description as unitary and then tell you something about future sequences of eventsgiven a history of past sequences of events. Computational modeling of collaboration needs a unit out of which tobuild up larger structures and interpretations. What are the appropriate units?

Many instances of collaboration are directed towards the construction of some artifact or achieving some state in theworld. The physical actions undertaken on relevant objects are candidates for the primitives of computationalmodels. However, it is desirable to have a level of analysis that abstracts away from domain-specific actions tosupport a theory that is applicable to many domains. Mühlenbrock & Hoppe (1999) provides an example of how todo this.

Since we are concerned with learning, the manipulation and interpretation of information is a plausible alternativelevel of analysis. My analysis in particular is based on a working definition of knowledge building as the accretion ofinterpretations on an information base that is simultaneously expanded by information seeking. This perspectivessuggests that computational modeling should be based on a level of description that takes acts of adding, modifying,merging or integrating information and referencing that information in acts of interpretation as unitary. Physicalactions should be abstracted up to this informational level

Since we are concerned with collaboration, we are particularly interested in how such accretion and interpretation ofan information base is accomplished between two people. Therefore we ask how such actions distribute acrossmultiple individuals yet collectively constitute knowledge building. We want to look at how information that iscontributed, manipulated, or interpreted by one person is subsequently taken up and manipulated or interpreted byanother person. I call such events “information uptake.”

Shared Representations

The present work focuses on modeling of collaborative learning with shared representations. Therefore, we shouldconsider how the model addresses actions in multiple representational media that may include persistent externalrepresentations. We’ll look at actions on an accreting knowledge base that constitute information uptake betweenparticipants, where these actions may involve multiple forms of representations (volatile and persistent; linguisticand graphical …). We’ll look for patterns in such actions that tell us something of interest about the collaboration. Inmy analysis of what people do with shared representations I have identified some patterns of interest.

The Study

The participants’ task was to propose and evaluate hypotheses concerning the cause of ALS-PD, a neurologicaldisease with an unusually high occurrence on Guam that has been studied by the medical community for over 50years. The experimental software (figure 1) provided a graphical tool for constructing representations of the data,hypotheses, and evidential relations that participants gleaned from information pages. An information windowenabled participants to advance through a series of textual pages presenting information on ALS-PD. The sequencewas designed such that later pages sometimes affected upon the interpretation of information seen several pagesearlier, making the use of an external memory important. In the study on which this analysis derives its data (Sutherset al., 2003), the software was modified for synchronous online collaboration with the addition of a chat tool. In mydiscussion, “verbal” refers to actions in the chat tool and “graphical” to actions in the graph.


43

Figure 1: Experimental Interface

The Analysis

I am conducting the analysis in a bottom-up manner, working from the referential level to the intentional level. Mystudent Ravikiran Vatrapu and I have identified ways in which information “flows” between participants through thegraph, as evidenced by their references to information in the graph, whether verbally or (more often) by directmanipulation. I am now layering on top of this analysis my own interpretations of the intentions behind thesereferences. In doing so, I am particularly looking for evidence of knowledge building, using a working definition ofknowledge building as the accretion of interpretations on an information base that is simultaneously expanded byinformation seeking. The act of interpretation may take the form of explicit sense-making commentary, but it mayalso take place through the transformation and integration of representations of the information base. Then,collaborative knowledge building takes place when multiple participants contribute to this accretion ofinterpretations by building, commenting on, transforming and integrating an information base. In undertaking theseinterpretations, I am also considering what certain theories of communication and group activity suggest that I lookfor, as discussed briefly below.

Participant’s actions in the graph as well as chat can be understood in terms of Clark’s model of grounding (Clark &Brennan 1991; Monk 2002). We can restate grounding in terms of actions on a representation: a participant expressesan idea in the representation; another participant acts on that representation in a manner that provides evidence ofunderstanding the first participant's intent in a certain way; the first participant can choose to accept this action asevidence of sufficient understanding, or, if the evidence is insufficient, initiate repair. Under the groundingperspective, the analyst would look for sequences of actions in which one participant’s action on a representation istaken up by another participant in a manner that indicates understanding of its meaning, and the first participantsignals acceptance. A problem for modeling is that this final signal of acceptance is often implicit, so be difficult toidentify, and can consist merely of continuing the interaction rather than initiating repair of a breakdown. Also, amodel based solely on grounding theory will not tell us much more than when participants have understood eachother or have identified a need to repair a misunderstanding. It would be more useful to identify patterns of discourseand reconstruct possible intentions. However, this perspective does suggest that we might view interaction throughrepresentations as a form of nonverbal or semi-verbal conversation. If we assume that contributions follow rules ofrelevance (Grice, 1975), we can than try to interpret actions on the representation under the assumption that they arerelevant to some previous action.


44

Under the socio-cognitive conflict perspective (Doise & Mugny 1984), we would want to identify situations in whichthe externalization of ideas led to identification of differences of interpretation that were subsequently taken up by atleast one of the individuals involved. A distributed cognition perspective (Hollan & Hutchins 2002) suggests thatcognitive activities such as knowledge building are distributed across individuals and information artifacts throughand with which they interact. Then, we would look for transformations of representations across individuals,especially if those transformations can be interpreted as an intersubjective cognitive process such as knowledgebuilding. Examples include merging, revising, and connecting representations of ideas. From the activity theoreticperspective (Bertelsen & Bødker 2003) I take the concept of mediation, and analyze collaborative use ofrepresentations by looking for ways in which the representation mediates (makes possible and guides) interactionsbetween participants by virtue of its form. This viewpoint is consistent with the distributed cognition perspective. Inaddition to the foregoing, I draw on my own ideas about how representations support collaborative activity by (1)initiating negotiation of meaning; (2) providing referential resources for conversation; and (3) providing thefoundation for mutual awareness of common ground (Suthers & Hundhausen 2003).

Examples and Their Implications

In this section I present examples of interaction through both the graph and the chat facility, and my interpretation ofthem. I begin by describing the notation used. In order to “see” how participants were interacting with each other, mystudent and I designed a mixed tabular/graphical representation of the sessions. This represents the activity (chat andchanges to the representation) of Participant 1 (P1) in the left hand column, and activity of Participant 2 (P2) in theright hand column. A column in the middle is reserved for annotations indicating “information uptake” relationsbetween actions. We use a diagrammatic notation for this information update. An arrow is drawn from action A1 toaction A2 if A2 builds on the information in A1. Examples include editing or linking to prior information, or cross-modal references such as a chat comment about an item in the graph. The arrow is directed from past to future, as itshows the "flow" of information between past and future actors (which may be the same or different participants) viathe representation. The links had to meet the criteria that the uptake identified is plausibly based on the informationalcontent or attitude towards that information of the uptaken act or representation. There must be evidence that theuptaker is responding to one of these. (For example, merely moving things around to make the graph pretty is notcounted.)

Example 1: Collaborating through the graph

The first example provides a basic example ofcollaboration through the graph leading to aconclusion that is acknowledged verbally. Thedetailed explanation provided for this example willalso serve to familiarize the reader with thenotations. To help ground this in what participantsworked with, the subgraph that resulted from thisinteraction is shown in Figure 2. (I added the labelson the boxes.)

The participants had previously represented ahypothesis (H02) that aluminum is the cause of thedisease, and two data items, D05 and D06, thesebeing linked by consistency (+) links D05+D06 andD05+H02. (Participants commonly use + to collectrelated data as well as for linking evidence tohypotheses.) After several pages concerning fading,they encounter a new page indicating that ALS-PDpatients have high levels of aluminum in theirbrains. The transcript begins at this point.

Figure 2: Portion of graph for Example 1


45

Example 1. Basic interaction and coming to agreement via the graph.

Previous Objects H02 Al or AlO is the causeD05 drinking water contains high levels of AlD06 from S. Guam

Context: Participants have just read page titled “High Concentrations of Aluminum Found in Diseased Brains” which states:“Neuropathologist Daniel Perl X-ray probed the brain tissue of some ALS-PD patients. He found unusually high concentrationsof aluminum in those brains. He says, "Normally, the background level of aluminum in a neuron is from one to three parts permillion. In the diseased Guam brains we're getting from three hundred to six hundred parts per million."”Time Who Act Object(s) Chat or graph content [spelling as given]15:49:51 A D26 ALS-PD patients have high Al concentration in brain15:50:20

P2A D27 normal Al level is 1:3 parts per million

15:50:35 P1 A D28 1-3 per million = normal15:50:51 A D29 ALS-PD Al level is 300:600 parts per million15:51:08

P2A D26+*D05

15:51:09 D D27 [deleted]15:51:26

P1M D28 Al level 1-3 per million = normal

15:52:21 P2 A D29+D2615:52:24 A D28+D2615:52:31 P1 M D29 Al level 300-600 parts per million ALS-PD brains15:52:47 P2 A D29+*H0215:52:52 P1 M D29 Al level 300-600 parts per million = ALS-PD15:53:25 M D05 drinking water contains high levels of Al in S. Guam15:53:29 D D06 from S. Guam

M [various] [repositions various objects for 44 seconds]15:54:13

P2

C boy we got something15:54:39 P1 C heheh ALUMINUM!!!!


46

Table 1: Codes used in examples

Acts: A Object added to representationC ChatD Object deleted from representationG Gesture on the indicated objectsM Object modified in representationS Spoken content

Objects: * Previously represented object is being reintroduced into the conversationD Data objectH Hypothesis object

Looking at the graphical representation of Example 1 (note that links “passing through” have been removed forreadability), we see that participants are collaborating through the graph (because there are solid lines other thanyellow), that this interaction involves consistency links (green), deletion (maroon) and revision (blue), and that itdraws upon material previously represented (lines going off the top of the image). Taking the arrow heads intoaccount, we see that in this segment P1 is taking responsibility for adding and editing the content of the text boxes,while P2 is linking together information contributed by both P1 and P2 (of four links, one involves only P1’smaterial, two bring P1 and P2’s material together, and one involves only P2’s own material.) Therefore this segmentexemplifies an asymmetric role division that was also seen in other pairs’ sessions. Stepping through this example,participants interacted as follows. P2 creates two data items D26 and D27 from the new information page. P1 isdoing so at the same time, creating D28, which is redundant with D27. While P2 continues to work, P1 recognizesthe redundancy, deletes P2’s version (D28) and rewords his or her own version D27 to include some informationfrom D28 (that it is about aluminum). Parallel redundant activity followed by merging and cleanup is common in ouronline transcripts.

Meanwhile, P2 goes on to add one more data item D29 and link it to D05. The manipulation of D05 is areintroduction of an item that has not been considered for a while: this exemplifies the utility of a visualrepresentation for reminding participants of previous information and enabling them to reference it easily. D05 wasoriginally created and was last manipulated by P1; therefore this incident also illustrates one participant taking upinformation that had previously been contributed by another (as indicated by the solid line).

Almost a full minute after P1’s deletion (they might have been absorbing what each other had just done), P2 linksD26 to both his or her own D29 and P1’s recent contribution D28, forming a cluster of related data. While P1 cleansup the wording of P2’s recent contribution (D29), P2 now makes the evidential relationship to the aluminumhypothesis H02 explicit – again performing a reintroduction of an item originally introduced by P1. P2 now startscleaning up in parallel to P1, by merging data items D05 and D06. After moving some things around to clean up thegraph, participants finally acknowledge verbally their shared interpretation of they have achieved though the graph:“boy, we got something”; “heheh ALUMINUM!!!!”

It is clear that participants were collaborating through the graph, taking up information that was introduced by theother participant, sometimes much earlier. Although the role distribution is asymmetric, the collaboration constitutesa simple form of knowledge building in which they use the graph notation to come to agreement on the structure ofevidence and its implication for a hypothesis under consideration.

Example 2: Disagreement

The interaction of Group 3 in example 2 exemplifies how a conversation-like interaction can take place throughmanipulation of the graph, and how conflict can be identified and addressed (albeit not satisfactorily in this case) viamanipulations of the graph. The relevant actions are abstracted in the table of Example 2, where they are annotatedwith their interpretation as a conversation (thus, this column of the table differs from that in the previous example).


47

Example 2: Group 3 engages in an argument through the graph (researcher’s interpretation).Unrelated discussion was removed at the thick line in the graphic, and unrelated annotation links are removed for

readability.

Context: Previously expressed by P1: H02 “(fading) cycad seeds in medicine cause guam diseases”Participants have just read page 12 titled “BMAA-fed Monkeys Exhibit Signs of ALS-PD” reading “When scientists fed largedoses of BMAA (an amino acid found in cycad seeds) to macaque monkeys, they observed the monkeys age before their eyes. Aftera few weeks' exposure to BMAA, some of the animals became weak. Over three months, some of the animals became apathetic,listless. Their hands trembled. They stooped and shuffled. Such symptoms are not unlike those of someone with ALS.”Time Who Act Object(s) Content (plaintext) or researcher’s interpretation (italics) of act14:35:40 P2 A D13 animals tested for BMAA an amino acid didn’t have the same … symptoms as

some one w/ als14:36:10 A D13?H02 I think that has something to do with H02, but I’m not sure what.14:36:23

P1D D13?H02 Never mind.

14:36:28 P2 A H02-D13 They conflict.14:36:36 P1 A D14 But it says that BMaa in cycad seeds14:36:51 P2 A D14+D13 | Right, that’s why.14:36:51 A D14+H02 | So it’s for the hypothesis.14:36:56

P1D H02-D13 You’re wrong.

Examining the reference annotations we see that both participants (see arrowheads) are bearing the burden ofcollaboration by integrating each others’ information (combinations of solid and dotted lines for three of the links).We also see that all polarities of evidential relations are being considered (grey, red, green). There is no chat in thissegment: in fact, participants only chatted one more time, several pages later, on an unrelated hypothesis

In this exchange, participants are exploring the implications of some new evidence for their second hypothesis(H02), that the cycad seeds cause the disease. Reading my interpretation down the right hand column of the table,this interaction has the form of a disagreement: P1 suggests the possibility of a relationship (D13?H02); P2 proposesa negative relationship (H02-D13); then, after introducing some new data (D14), P1 proposes a positive relationship(D14+H02, where D14 is linked to D13), and deletes P2’s proposed relationship. At the same time, P2 is using thenew data P1 introduced to support his or her own interpretation (D14+D13). Participants are clearly engaging in aform of argumentation through the graph, without using the chat tool.

Upon closer examination the source of the disagreement can be seen to be an erroneous reading of the text. The textcontains a double negative” “Such symptoms are not unlike those of someone with ALS.” P2 apparently read this assimple negation, writing that the animals “didn’t have the same symptoms as some one w/als”. This error accounts


48

for P2’s confidence that the data conflicts with H02. Apparently, the participants did not identify the source of theirdisagreement in this error of interpretation.

Examples 3, 4: The Roles of Chat and Graph

With a few exceptions at the end of the sessions, most of the task-oriented interaction took place through the graph.However, chat at times played a crucial role in supporting the communication. There was also one session in whichparticipants discussed what to do extensively in the chat.

The graph, naturally, was used primarily for what its representational primitives support: reporting and recordinginformation gleaned from the source pages, proposing hypotheses, and indicating consistency/inconsistencyrelationships between these items; and was the primary means of accomplishing these communications, althoughthere are a few examples of chat that could have been accomplished via the graph medium.

Some pairs used chat primarily for social banter as they carried out task-oriented interactions in the graph. Typicallythis social use of chat was occasionally replaced with task-oriented chat, such as those exemplified in the nextsections. Task coordination included role assignments (“you do this one, OK?”) and coordination of page turning(“ready?” “next?”, etc.). Occasionally, brief chat exchanges during the session would focus on the value orinterpretation of information.

Two examples from group 3 follow. In Example 3, P2 is questioning whether to bother with some new information.

Example 3: Commenting on the value of information

Example 4: Moving to chat to reflect on unexpected information

Example 4 is the portion omitted from the figure for Example 2. Note that the chat is summarized by P1’s action inthe graph.


49

Interestingly, several groups engaged in extended evaluative / interpretive discussions after reaching the final page,which announced that participants’ “library research” was done. Analysis of these conversations to infer how theymight be making use of the visual representation is on my future agenda.

Discussion of Implications for Modeling Collaboration

The foregoing examples showed how episodes of collaboration can take place primarily and even exclusively viamanipulations of a visual knowledge-representation medium (at least for a span bounded by initial and concludingverbal comments). Collaboration can also be distributed across multiple representational modes of communication.

Interpreting Actions in a Shared Workspace

One implication of this fact for modeling is that models of collaboration should be able to interpret actions onrepresentations in shared workspaces as actions of communication and negotiation – as if they were a conversation –as well as interpreting them in terms of their domain-specific semantics. Mühlenbrock & Hoppe (1999) have madethis point, showing how actions in a workspace for a card-matching problem can be interpreted in terms ofidentifying and addressing conflicts between proposed solution components.

My own examples illustrate the danger of taking a superficial approach to mapping domain level actions tointentions. Specifically, suppose we wanted to write a pattern for recognizing “disagreement” about two alternativeinterpretations of data. A simple pattern would be: P2 makes a proposal; P1 deletes that proposal and replaces it withanother one. (This pattern is similar to that used by Mühlenbrock & Hoppe). Example 2 in this paper shows thatsome variations on this pattern will be necessary. In that example, we have a partial match to the pattern justexpressed: P2 proposes an evidential interpretation H02-D13, and subsequently P1 deletes it. However, there is nodirect replacement of H02-D13 with another link (such as H02+D13). Instead, to understand this sequence we mustrecognize that P1’s alternate proposal is H02+D14, and D14 is somehow related to D13 (as expressed by P2’s – notP1’s! – D13+D14). Furthermore, the sequence begins not the proposal that is presented by P2 and deleted by P1, butrather when P1 suggests that there is an evidential interpretation of interest (H02?D13), but then withdraws it.

We might be tempted to conclude that the central indicator of disagreement via actions in a workspace is that oneparticipant deletes something the other participant has contributed: replacement with an alternative is not necessary.The first example demonstrates the inadequacy of this simplification, and further reinforces the importance of adeeper understanding of the actions undertaken. In Example 1, P2 adds D27 and then P1 subsequently deletes it.Superficially this also looks like disagreement, but in actuality the participants are very much in agreement. P1 hadadded D28 at about the same time as P2’s D27, and they express nearly the same fact. To properly understand P1’sdeletion of P2’s as a component of an act of agreement rather than disagreement, we must know that the contents aresimilar and that P1 subsequently modifies his/her own D28 to be closer in wording to that of the deleted D27.

Interpreting Actions Across Modalities

Actions can flow between different representational modalities, e.g., between verbal and graphical in my data. Thisfact has two implications for computational modeling:

The first is that abstraction from modality-specific actions is necessary: Computational models must be able toabstract from the medium of interaction, interpreting actions such as information uptake even if they occur acrossmultiple representations. For example, in Example 1 the final chat exchange “boy we got something”, “heheALUMINUM!!!” is not interpretable in itself, but rather as an acknowledgement of what was just achieved throughthe interaction in the graph. We can only know that the “something” is aluminum as a possible cause for the diseaseby looking at the actions in the representation. In Example 3, we are actually dealing with three representations. Thecomment about the value of information can only be understood by knowing that both participants have moved to anew information page; P2’s subsequent dismissal of his/her own comment can only be understood by seeing that P1has implicitly indicated the relevance of the information through the addition of D10 to the graph.

The second point is that modality shifts may be significant: abstraction can leave out critical information:Computational models should not ignore the medium in which an action is taken: something will be lost if the model


50

works only from abstracted descriptions. In my data the acts of moving from verbal to graph or back again aresignificant in themselves. I have several examples where collaborators move from the verbal to the graphical mediato carry out an agreed upon action: the movement to action in the graph signals acceptance of a proposal justdiscussed. Therefore, a computational model that is intended to recognize events at the intentional level shouldconsider a shift from a lesser-constrained medium such as natural language to action in a problem specific workspaceas a potential instance of acceptance of a proposal.

Example 1 illustrates the second point in the other direction: participants in Example 1 move from graph to chat toconfirm their mutual interpretation of the collaborative action just undertaken in the workspace. Collaborators maymove from graph to verbal when an unexpected or difficult situation comes up: a critical event in which the richnessof the verbal medium is needed for explicit interpretation and negotiation. Example 4 is also one such case.Participants have encountered information that causes difficulty for their previous hypothesis. This is so unexpectedthat they shift to a chat interaction, even though some of the contents of this chat could have been accomplished inthe graph (e.g., by creating a data item for the abundance of aluminum and linking it with a minus link to thealuminum hypothesis). I have other examples in my data where unexpected data leads to a level of discussion in thechat that is not expressible in the graph. In both cases, the shift to chat is itself a signal that something unexpectedhas been encountered. A computational model of collaboration that is intended to recognize critical events shouldlook for joint reflective activity whenever a mode shift is made from a constrained representation to natural language.

In general, while being able to interpret actions in terms of information manipulation and interpretationindependently of their representational mode, computational models must also consider mode shifts as potentialevidence for critical events in the collaborative process.

References

Aimeur, E., Dufort, H., Leibu, D., Frasson, C., Some justifications about the learning by disturbing strategy, AI-ED97, Japan (1997) 119-126.

Bertelsen, Olav W. and Bødker, Susanne (2003). Activity Theory. In J. M. Carroll (Ed.), HCI Models, Theories andFrameworks: Towards a Multidisiplinary Science. San Francisco, Mogan Kaufmann: 290-???.

Chan, Tak-Wai (1996). Learning companion systems, social learning systems, and the global social learning club.Journal of Artificial Intelligence in Education 7(2): 125-159.

Clark, H.H. & Brennan, S.E. (1991). Grounding in Communication. In L.B. Resnick, J.M. Levine and S.D. Teasley(eds.), Perspectives on Socially Shared Cognition (pp. 127-149). Hyattsville, MD: American PsychologicalAssociation.

Constantino-Gonzalez, M.A., Suthers, D.D. and Escamilla de los Santos, J.G. Coaching web-based collaborativelearning based on problem solution differences and participation. International Journal of Artificial Intelligencein Education, 13, 2002.

Doise, W., and Mugny, G. (1984) The Social Development of the Intellect, International Series in ExperimentalSocial Psychology, vol. 10, Pergamon Press

Festinger, L. (1957). A Theory of Cognitive Dissonance, Stanford University Press, 1957.Grice, H. P. (1975). Logic and Conversation. Syntax and Semantics. P. Cole and J. L. Morgan. New York, Academic

Press. 3: 41-58.Hollan, J., E. Hutchins, et al. (2002). Distributed Cognition: Toward a New Foundation for Human-Computer

Interaction Research. Human-Computer Interaction in the New Millennium. J. M. Carroll. New York, ACMPress Addison Wesley: 75-94.

Corbett, A. T., Koedinger, K. R., & Anderson, J. R. (1997). Intelligent tutoring systems. In Helander, M. G.,Landauer, T. K., & Prabhu, P. V. (Ed.s) Handbook of Human-Computer Interaction, (pp. 849-874).Amsterdam, The Netherlands: Elsevier Science B. V.

Monk, A. (2003). Common Ground in Electronically Mediated Communication: Clark's Theory of Language use. InJ. M. Carroll (Ed.), HCI Models, Theories and Frameworks: Towards a Multidisiplinary Science. San Francisco,Mogan Kaufmann: 265-289.

Mühlenbrock, M., & Hoppe, U. (1999). Computer Supported Interaction Analysis of Group Problem Solving. InProceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference, C. Hoadley & J.


51

Roschelle (Eds.) Dec. 12-15, Stanford University, Palo Alto, California. Mahwah, NJ: Lawrence ErlbaumAssociates.

Soller, A., & Lesgold, A. (2003). A Computational Approach to Analyzing Online Knowledge Sharing Interaction.In Artificial Intelligence in Education, H. U. Hoppe, F. Verdejo & Judy Kay (Eds.) Amsterdam: IOS Press(Proceedings of 11th International Conference on Artificial Intelligence in Education: AI-ED 2003), pp. 253-260.

Suthers, D., Girardeau, L. and Hundhausen, C. (2003). Deictic Roles of External Representations in Face-to-faceand Online Collaboration. Designing for Change in Networked Learning Environments, Proceedings of theInternational Conference on Computer Support for Collaborative Learning 2003, B. Wasson, S. Ludvigsen & U.Hoppe (Eds), Dordrecht: Kluwer Academic Publishers, pp. 173-182..

Suthers, D., and Hundhausen, C. (2003). An Empirical Study of the Effects of Representational Guidance onCollaborative Learning. Journal of the Learning Sciences, 12(2), 183-219.


52

Helping Groups Become Teams: Techniques for Acquiring and Maintaining Group Models

Patrícia Azevedo Tedesco1, Marta C. Rosatelli2

1Centro de Informática, Universidade Federal de Pernambuco Av. Prof. Luiz Freire s/n, Cidade Universitária, Recife-PE, 50740-540, Brazil

2Programa de Pós-Graduação em Informática, Universidade Católica de Santos R. Dr. Carvalho de Mendonça, 144, Vila Mathias, Santos-SP, 11070-906, Brazil

Email: [email protected], [email protected]

Abstract: People are small group beings. Interacting with group members provides us with the opportunity to receive feedback, to discuss different ideas, and to get support for our endeavours. Groups generally learn faster, make fewer errors, recall better, make better decisions and are more productive than individuals working on their own. However, not all groups achieve high performance. Typically, groups start off as being traditional groups and in some cases evolve to teams, where the group performance is outstanding. In order to assist their groups in the process of evolving to teams, collaborative/cooperative systems need to keep a model of their group. In this work we present a review of different AI techniques and tools that implement them, presenting their benefits and difficulties regarding the user and group modelling tasks. The aim is to point out open questions related to choosing which technique use to model the group in different circumstances in order to help groups become teams.

Keywords: Group modelling, collaborative systems, teams

1 Introduction

People are small group beings. In fact, we seem to spend most of our daily lives as part of one group or another. When interacting with group members, we have the opportunity to receive feedback, to discuss different ideas, and to get support for our endeavours. Consequently, groups generally learn faster, make fewer errors, recall better, make better decisions and be more productive than individuals working on their own (e.g. Baron et al., 1992). This is due to what has been called process gain – members’ interaction brings out insights that none of the group participants had envisaged before. Further explanations for this phenomenon include groups’ quicker recognition of incorrect solutions, and their keeping of a better memory of what has happened during the interaction.

However, research in Organisational Psychology has shown that not all groups achieve high performance. In typical work and/or learning scenarios groups start off as being traditional groups (where members agreed to work together but do not see much benefit in doing so) and in some cases evolve to teams, where the group performance surpasses the sum of the individual performances. In order to assist their groups in the process of evolving to teams, collaborative/cooperative systems need to keep a model of their group (individuals, roles played, agreements, quality of interaction) and use it as a resource for providing support in such scenarios. In this light, researchers in Computer Supported Collaborative Learning (CSCL) and Computer Supported Cooperative Work (CSCW) have started to investigate how to adapt the techniques used for individual user modelling in order to acquire and maintain group models.

In this work we present a review of the research being done about Artificial Intelligence (AI) techniques for user and group modelling. Our review aims at pointing out open questions in the area concerning recommending which AI technique is most suitable to model the group in different circumstances in order to help groups become teams. Such recommendations can provide both designers and new researchers in the area with guidelines for their user and group modelling tasks.


53

In the context of group modelling, there are other approaches for group modelling, which are based on Psychological theories. Such models can inform the type of computational analysis that needs to be done for the acquisition of group models. However, it is not always simple to translate the psychological models into information that can be captured/processed by the machine. Consequently, there is a strong need for AI techniques. Amongst such theories, Leontiev’s Activity Theory (AT) is beginning to be used in collaborative systems. This theory is used as a tool to investigate the development of human activities mediated by context and socio-cultural artefacts. According to the AT’s principles is through human activities, mediated by cultural artefacts and people, that consciousness are developed. The usage of Leontiev´s AT gives the designer a good insight into how the tasks are divided among system and users. Besides, AT also shows us which kinds of perceptions the system must have in order to act in a timely fashion. DEGREE (Barros and Verdejo 2000) is an example of a system that uses the AT to analyse the group’s interaction and maintain the group model. However the review and discussion of such techniques is out of the scope of this paper.

This paper is organised as follows. Section 2 presents an overview of the Artificial Intelligence (AI) techniques currently used for acquiring and maintaining group models. Section 3 presents some examples of systems that maintain group models. Section 4 discusses how to choose the appropriate technique for the different situations. Finally, section 5 presents our conclusions and suggestions for further work.

2 AI Techniques for Group Modelling

Below we present a review of AI techniques for individual user modelling, which are also applied to group modelling. The objective is to highlight the different nuances of the techniques that are most widely used.

2.1 Logic

Logic-based user modelling is commonly used together with other techniques. Typically, logic-based user models are considered to be a set of beliefs. From a logic-based user model, inferences can be made about users’ beliefs, and may result in complex beliefs as well. However, in order to better provide for adaptivity, sometimes systems need to keep different information in their user models (e.g., plans, goals, skills). According to Pohl (1999), the most common types of sentences in logic-based user models are: beliefs, preferences, interests, and desires. However, other operators can be defined as well.

Pohl (1999) states that there are two major approaches in logic-based user modelling: partitions and modal logic. In the first category, different types of sentences (e.g., goals and beliefs) are stored separately. Modal logics allows for the introduction of several types of operators to characterize system’s assumptions. The most commonly used are possibility (◊) and necessity ( ). Modal logics allows the system to represent negative assumptions, (System believes that the user does not know Sis B ¬ Usr B p), which cannot be represented in the partition-based approach. When representing only contents in the user model, the most common representations are: (1) propositional calculus; (2) first order predicate calculus; and (3) other specific formalisms, like frames and semantic networks.

Logic based models are still frequently used, specially when keeping track of the user’s mental state (beliefs, intentions, goals) is important. It should be noted, however, that complex logic-based models are inefficient and hard to maintain.

2.2 Machine learning

Recently, the use of machine learning techniques for user modelling has shown to be more appropriate than traditional knowledge-based methods, especially in applications that comprise large or difficult-to-specify domains. The reason is that these techniques do not need a demanding knowledge acquisition process to make predictions about the user (Kobsa 2001).


54

Machine learning is suitable for modelling both individuals and groups: while many of the existing applications model individual users, emergent applications are related to generic models of communities of users (e.g., as in e-commerce). Also, it is appropriate for applications in which the user performs a task that involves repeatedly selecting between pre-defined options. In this case, data about the user behaviour would generate a set of training examples that could be used by the system to predict his or her future actions. On the other hand, user modelling presents characteristics that make the direct application of machine learning techniques difficult: the need for large data sets, the need for labelled data, concept drift, and computational complexity (Webb et al., 2001).

2.3 Plan recognition

An important aspect of user modelling is to identify which are the user plans and objectives. Plan inference systems start with a set of objectives that an agent might pursue and an observed action that was carried out by this agent, and have to infer the agent objective and to determine how the observed action contributes for this objective. In most real situations, chaining in plan recognition produces multiple hypotheses about the agent’s plans, and the plan recognition has to deal with techniques that reduce the space of viable hypotheses. In the context of user modelling, this means that systems can also request specific information to support decisions about the user plans ambiguities for the purposes of the current interaction (Carberry, 2001).

Bayesian Networks (Pearl, 1988) have been used for dealing with uncertainty in plans inference and offer a promising approach for plan recognition in situations where enough training data can be collected. Dynamic Belief Networks are also used as they capture the influence of temporal aspects (Jameson et al., 1999). On the other hand, there are a few problems in plan recognition for user modelling. In Bayesian Networks, identifying an appropriate network structure remains a drawback. Noise in the data is another problem, which distorts the system’s ability of identifying with precision the agent’s plans. Also, any problem occurred in the three kinds of inputs of this type of system (library of plans, input of the agent whose plan is being recognized, and any partial plan that already has been inferred) can make the system infer an incorrect plan or infer no plan at all. Finally, regarding robustness of plan recognition systems, the problems are intensified when inferring student’s plans in Intelligent Tutoring Systems (ITS). In this kind of system the student is not qualified in the domain and thus tends to make new types of errors (Carberry, 2001).

2.4 Uncertainty and probabilistic reasoning

More often than not, user modelling must deal with uncertainty regarding inferences made about a user in the absence of complete information. Machine Learning and reasoning under uncertainty comprise a variety of techniques that together form predictive statistical models (e.g., Decision Trees, Neural Networks, and Belief Networks). The use of predictive statistical models for user modelling is relatively recent, and was stimulated by both the great amounts of available electronic data and advances in machine learning, and has been used to adapt the behaviour of a system. These include: Linear Models, TFIDF (Time Frequency Inverse Document Frequency)-based Models, Markov Models, Neural Networks, Non-supervised Classification Methods, Rule-induction Methods, and Bayesian Networks (Zukerman and Albrecht, 2001).

3 Examples of Group Modelling in Collaborative Systems

In order to be able to foster more productive interactions, various current collaborative learning systems maintain group models as a basis for monitoring the interaction and thus providing both general and user-specific feedback. In this section, we discuss a few of the current approaches found in the literature.

SmartChat (Siebra et al., 2004) is an intelligent environment for collaborative discussions. It uses an argumentation model to organise users’ interactions and it classifies users in pre-defined stereotypes, based on their participation in the discussion. That classification helps the SmartChat's agent society to interfere in the discussion, in order to motivate users’ participation, and also recommend references or stimulate pair collaboration. The system uses a logical-based representation for its group model.


55

OXEnTCHÊ–Chat (Vieira et al., 2004) is a generic chat tool coupled with an automatic dialogue classifier that analyses on-line interaction and provides just-in-time feedback to both instructors and learners. The system keeps a logic-based group model based on the system’s classification. This model is then used by a bot agent, that can work by coordinating the dialogue, or providing content where needed.

COLER (COllaborative Learning environment for Entity-Relationship modelling) (Gonzales and Suthers, 2002) is an Internet-based collaborative learning environment. Students first work individually (in their private workspace) and then collaborate to produce an Entity-Relationship (E-R) model. Each student has a personal automated coach. It gives feedback to the student whenever a difference between his and/or her individual E-R models and the one built by the group is detected. The coach uses Decision Trees (Mitchell, 1997) in order to decide how to present feedback.

COMET (A Collaborative Object Modelling Environment) (Soller et al., 2002) is a system in which teams can collaboratively solve object-oriented design problems, using the Object Modelling Technique (OMT). The system uses sentence openers in order to analyse the ongoing interaction. The chat log stores information about the conversation, such as date, day of the week, time of intervention, user login, and sentence openers used. COMET uses Hidden Markov Models to analyse the interaction and assess the quality of knowledge sharing and is an example of the use of predictive statistical models in collaborative systems (Soller and Lesgold, 2003).

MArCo (Mediador Artificial de Conflitos) (Tedesco, 2003) counts on an artificial conflict mediator that monitors the dialogue and detects conflicts, giving tips on how to better proceed to the participants. The mediator uses a Belief-Desires-Intentions (Michael and Georgeff, 1993) model in order to reason about the dialogue and decide on how to intervene. MArCo’s interventions are restricted to the moments where a conflict has been detected. This system provides an example of a logic-based group model.

In summary, there is a definite research trend concerned with exploring group models in order to better support collaborative interaction. At this point in time, most researchers in the area are still experimenting with different approaches and AI techniques to group modelling, that generally either concentrate on the individual or on the group. In order to help groups become teams, it is necessary to provide good support for both extremes.

4 Choosing the Adequate Technique for Different Applications

Choosing the most adequate AI technique for modelling a group is not a trivial task. Such a choice seems to depend on different levels of the application design. Factors that can affect group modelling are, among others, the system architecture (e.g., traditional or agent-based); user interface features such as the kind of (graphical) representation used (if any); and the mode of interaction – graphical or natural language. Also, this design task (choosing the appropriate AI technique) appears to rely on several characteristics of the collaborative system itself, such as the knowledge domain; the pedagogical approach, that in a final analysis define both the type and what is expected from the collaborative interaction; the collaboration resources (e.g., chat, discussion lists, and so on); the variables that define the degree/quality of collaboration (e.g., number of contribution or kind of contributions); and so on. Together, all the factors mentioned above, define what should and/or can be acquired to build the group model and how this model will be updated in order to provide a system response that fosters productive interactions and, more importantly, helps groups becoming teams.

Due to the accelerated development of the web in the last years, quite a few educational collaborative systems that adapt their behaviour to the group have been developed. This seems to indicate a trend, and the possibility of using usage data, besides user and/or group data for group modelling is significant. And, in this context, a hybrid user and group model seem, at first, to be more appropriate. Particularly regarding web-based systems, in which the huge amount of available data is a common feature, machine learning and/or uncertainty and probabilistic reasoning are suitable techniques, and can be used alone or combined with logic-based models.

An example of a collaborative system that has a hybrid user model that combine machine learning and knowledge-based techniques for representation and inference is given in (Gaudioso and Boticario, 2003). The user model is constructed and dynamically updated from user and usage data. The challenge is to implement these ideas to adapt the system to the group rather than to the individual user, which now is an interesting open research question in group modelling. In fact, the collaborative filtering (or clique-based) (Koychev and Schwab


56

2000) approach suits this type of task. Collaborative filtering is applied when a user behaves in a similar way to other users so that data from a group of users can be used to make predictions about individual users.

5 Conclusions

The recent advances in Information and Communication Technologies have brought to light the possibility for people to maintain group interactions even when at a distance. However, in order to provide for more productive interactions, researchers in CSCL and CSCW have investigated how to adapt the techniques used for individual user modelling in order to acquire and maintain group models.

In this work we have presented a review of the different AI techniques that have been used for user modelling, pointing out situations where the techniques would seem most appropriate to group modelling. When deciding which technique to apply, several questions must be addressed: (1) which tasks is the system providing support for (interface vs learning)?; (2) what is the system’s architecture (traditional vs agent-based); (3) mode of interaction;(4) goals of the collaborative system; (5) available resources. These define what type of knowledge (and thus the technique) that must be used to acquire and maintain the group models.

The availability of low cost communication and information technologies has brought along the challenge of building user/and group models that are accurate, provide enough knowledge so that system’s provide support in the correct contexts, are small and easy to maintain, and do not distract the user from their task. In summary, current research points out to hybrid models and is concerned with finding the combination of AI techniques that fulfil the requirements above.

References

Baron, S., Kerr, N., and Miller, N. (1992). Group Process, Group Decision, Group Action. Pacific Grove, CA: Brooks/Cole. Barros, B. and Verdejo, M. F. (2000). Analysing student interaction processes in order to improve collaboration.

The Degree Approach. International Journal of Artificial Intelligence in Education 11, 221-241.

Carberry, S. (2001). Techniques for plan recognition. User Modeling and User-Adapted Interaction 11(1-4), 31-48.

Gaudioso, E. and Boticario, J. G. (2003). Towards web-based adaptive learning communities. In U. Hoppe, F. Verdejo, and J. Kay (Eds.), Artificial Intelligence in Education, pp.237-244. Amsterdam: IOS Press.

González, M. A. C. and Suthers, D. D. (2002) Coaching collaboration in a computer-mediated learning environment. In Proceedings of Computer Support for Collaborative Learning 2002, pp. 583-584. Hillsdale: Lawrence Erlbaum Associates.

Jameson, A., Schäfer, R., Weis, T., Berthold, A., and Weyrath, T. (1999). Making systems sensitive to user’s changing resource limitations. Knowledge-Based Systems 12, 413-425.

Kobsa, A. (2001). Preface. User Modeling and User-Adapted Interaction 11, 1-4.

Koychev, I. and Schwab, I. (2000). Adaptation to drifting user’s interests. In Proceedings of ECML 2000 Workshop: Machine Learning in New Information Age. Barcelona, Spain.

Leontiev, A. N. (1978). Activity, Consciousness, and Personality. Hillsdale: Prentice-Hall.

Mitchell, T. M. (1997). Machine Learning. New York: McGraw-Hill.


57

Pearl, J. (1998). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann Publishers.

Pohl, W. (1999). Logic based representation and reasoning for user modeling shell systems. User Modeling and User Adapted Interaction 9, 217-282.

Rao, A. S. and Georgeff, M. P. (1991). Modeling rational agents within a BDI-architecture. In R. Fikes and E. Sandewall (Eds.), Proceedings of Knowledge Representation and Reasoning (KR&R-91), pp. 473-484. San Mateo, CA: Morgan Kaufmann Publishers.

Siebra, S., Christ, C., Queiroz, A., Tedesco, P., and Barros, F. (2004). SmartChat: An Intelligent Environment for Collaborative Discussions. Accepted for poster presentation at the 7th International Conference on Intelligent Tutoring Systems.

Soller A., Wiebe J., and Lesgold, A. (2002). A machine learning approach to assessing knowledge sharing during collaborative learning activities. In Proceedings of Computer Support for Collaborative Learning 2002, pp.128-137. Hillsdale: Lawrence Erlbaum Associates.

Soller A., and Lesgold, A. (2003). A computational approach to analyzing online knowledge sharing interaction. In U. Hoppe, F. Verdejo, and J. Kay (Eds.), Artificial Intelligence in Education, pp.253-260. Amsterdam: IOS Press.

Tedesco, P. (2003). MArCo: Building an artificial conflict mediator to support group planning interactions. International Journal of Artificial Intelligence in Education 13, 117-155.

Webb, G. I., Pazzani, M. J., and Billsus, D. (2001). Machine learning for user modeling. User Modeling and User-Adapted Interaction 11(1-4), 19-29.

Vieira, A., Teixeira, L., Timóteo, A, Tedesco, P., and Barros, F. (2004). Analyzing On-Line Collaborative Dialogues: The OXEnTCHÊ–Chat, accepted for publication at the 7th International Conference on Intelligent Tutoring Systems.

Zukerman, I. and Albrecht, D. W. (2001). Predictive statistical models for user modeling. User Modeling and User-Adapted Interaction 11(1-4), 5-18.


58

http://www.cogs.susx.ac.uk/ijaied/abstracts/Vol_13/tedesco.html

Designing Mechanisms to Stimulate Contributions in Collaborative Systems for Sharing Course-Related Materials

Julita Vassileva, Ran Cheng, Lingling Sun, Weidong Han Computer Science Department, University of Saskatchewan,

57 Campus Drive, Saskatoon, S7N 5A9 Canada {jiv, rac740, lis102} @ cs.usask.ca

Abstract This paper presents several mechanisms for stimulating contributions from students in a peer-to-peer environment. The evaluation of a status and visualization based mechanism showed a significant increase in student participation and contributions, but a decline in the quality of contributions. Keywords: motivation, participation, persuasion, collaboration, peer-to-peer, evaluation

1. Introduction Peer-to-peer (P2P) systems have a lot of potential uses in classroom and academic environments. The concept of P2P assumes decentralization and equality of participants: there is no central provider or services or material, but the users who consume the services or materials are also the providers of services and materials. Before the emergence of specialized P2P protocols for file-sharing, like NAPSTER, Gnutella, Chord, etc., the concept of P2P has been applied in collaborative learning environments, like Phelps (Greer et al, 1997), I-Help (Vassileva et al., 1999), where learners can ask for help their peers and receive help from their peers. Phelps was implemented with a client-server architecture, while I-Help was implemented using a multi-agent architecture and was initially fully decentralized. Later versions were centralized (all agents ran on a powerful server with an Oracle database). More recently, P2P file-sharing protocols have been used to design shared but fully distributed learning object repositories. For example, Edutella (Neidl, et al. 2002) is based on the JXTA platform. Comtella (Vassileva, 2002) uses the Gnutella protocol and allows graduate students to share research papers that they have found on the web and stored locally on their disks. P2P systems are fairly easy to build; they do not require any sophisticated models of collaboration, since they are mostly based on loose cooperation by the users. Users typically share resources in an asynchronous way and do not engage in dialogues or collaborative problem solving; these systems provide only the infrastructure, search and match-making, they do not facilitate explicitly the knowledge building process. There is no need for creating a pool of resources, or knowledge-base bootstrapping, as in other systems; the users provide the resources themselves. Yet, these systems are collaborative by nature, i.e. they can not exist without user cooperation, i.e. good will to share resources, to participate in the system, be on-line and answer requests. This is a much broader definition of collaboration than the one typically used in the area of CSCL, but even ensuring this level of collaboration is not straightforward. The lack of user participation has been the main problem in most such systems. It has been a problem also in online communities where statistics show that most users are lurkers or free-riders. Designing a successful on-line community is hard and most attempts fail. Only few evolve into sustained productive groups. In the next three sections we present our experience in designing mechanisms encouraging participation on different levels in a P2P system.


59

2. Ensuring presence P2P systems depend very much on who is on-line at the moment, i.e. how many users have their servents running. A servent that is not running can not respond to queries, and the resources / services shared by the user are unavailable to the other users in the system. In addition, in some protocols, like Gnutella, servents participate in forwarding queries of other users, i.e. they are part of the infrastructure. If they are unavailable, parts of the infrastructure are unavailable too, and messages can not be transferred. Therefore, ensuring that users keep their servents running is important. However, this turned out to be one of the main blocks on the way to deploying I-Help and Comtella. In the deployment of both systems, we observed that students activate their servent only when they needed help or searched for some paper. There are good reasons for this behaviour: having an extra application running all the time takes computing resources and space on the screen (even when the application icon is shrinked down on the task bar or running in the background). Unfortunately, this selfish behaviour makes it impossible for the community of simultaneous users to reach a critical mass to allow for everyone to find what they are looking for. As a result, students who try the system and don’t find anything won’t log in again and they are lost (together with their resources) for the community. There is a simple technical solution to avoid such negative feedback loop – to move all the user servents (or agents) on a server, where they can be running all the time and let users log-in to their own agents or servents when they need to search for resources or services themselves. This solution worked well in the case of I-Help. We applied it also in an adaptation of the Comtella system to support the “Ethics and IT” class (Vassileva, to appear) and even though it is a compromise with the architectural purity of the system (since it imposes a centralization), we argued with ourselves that it is only an implementation issue, since on a higher level the system remains distributed. All the servents communicate using the Gnutella protocol, i.e. any subset of servents can be moved to different servers, or back to the user computers and they will still work in the same way. Hybrid solutions, where some of the peers reside on user computers, if the users are willing to keep them running, and some reside on servers (where users can not ensure that they will keep their servents running) are also possible. In this way, by placing some or all the servents on a server, the basic level of participation (keeping the servent running) was ensured, therefore, access to all shared resources of all peers was ensured all the time. 3. Ensuring regular new contributions A P2P file-sharing system without new materials shared regularly by at least some of the users quickly reaches equilibrium: all users have already found and downloaded the files they are interested in. If they don’t find anything new of interest when they search several times, they won’t try again and will loose interest in the system. Maintaining a constant stream of new contributions (or shared materials) is very important for the success of the system. However, this problem can not be solved so easily as the problem of ensuring presence. To contribute new resources, the users need to invest some effort, including finding the new material, annotating it, performing the needed actions to share it with other peers. We applied several different methods of motivating users to make contributions. 3.1. Convenient interface for sharing new resources – “zero-effort” sharing For example, a pop-up window can appear in the browser reminding the user to share the file, if the user spends a certain amount of time on a web-page or if she prints a page, or if she is looking at a PDF document. The window can automatically suggest annotation for the resource using tools like semantic classification of the document based on the text, URL etc. We applied such proactive reminder in a previous version of the Comtella system for sharing academic papers, however some of the users found it somewhat too intrusive. If activated only for a certain type of files (PDF and PS, in which typically research papers are available on the internet), it is more acceptable. However, in our “Ethics and IT” class the resources to be shared were articles appearing on web-journals, like www.wired.com, www.itbusiness.ca, the web-edition of New York Times, Atlantic Monthly, or on discussion


60

http://www.wired.com/

http://www.itbusiness.ca/

forums, on-line communities like Slashdot.org, etc. The format of files from such sources is too heterogeneous and automatic prompting (without using heavy semantic classification tools) is unlikely to be successful: it will either omit too many files or it would be annoying. 3.2. Rewarding contributions Reciprocation is a basic norm in human society [4]. Reciprocation, through rewarding users for sharing files has been used in existing P2P filesharing systems. Some approaches introduce the notion of artificial currency and micro-payments, i.e. create a virtual market. We attempted this approach earlier (Vassileva et al., 1999, Kostiuk & Vassileva, 1999) in I-Help, but it didn’t stimulate much participation (Vassileva, Deters, 2001). Partially this was due to the inappropriate way our e-currency was “cashed” in the experiment. Instead of cashing it in grades (e.g. participation marks), we distributed souvenirs to the top users, which didn’t thrill the students. However, before attempting a second time, in a class that was in our control to award participation marks based on accumulated currency, the fate of Mojo-Nation and Clay Shirky’s article on micro-payments (Shirky, 2003) convinced us that any payment-based approach will not work because the act of “buying” a resource, even at a very small price, creates mental transaction costs, that is, energy required to decide if the resource is worth buying or not. There are other approaches to rewarding participation. KaZaA rewards active users with a better quality of services. The system records the actions of users and maintains a numeric participation level for each user. The speed of downloads the user can get is based on this value. We applied in a previous version of Comtella (Sun et al., 2003) a similar idea, but it turned out that a better download speed was not seen by users as a particular reward, since the speed was very good anyway, the files were relatively small and a couple of seconds didn’t matter. In the “Ethics and IT” version of Comtella, we rewarded user participation with access to more powerful search options (remove duplicate results, sort results by different criteria, show only new resources, shared after the user’s last logout, etc.). While the data from the usage of these options as well as the user questionnaire showed that these extra-functions were valued by the users, they could have acted as a double-edged sword, since they empowered the users who were already actively using the system. Perhaps allowing these functions for users who didn’t participate actively would have made the system more attractive for them. 3.3. Social Visibility One fundamental way how people decide what to do in a situation is to look at what other people are doing or have done. If many individuals have decided in favour of doing something, more people will tend to follow their way. Moreover, a group of people sharing some similarity can influence each others’ behaviour more effectively (Cialdini, 2001). According to the social validation theory (Cialdini, 2001), it would be possible to persuade people to make contributions to a P2P community by demonstrating that many people just like them have contributed a lot to the community and benefited from their contributions. Visualizing the community and the level of contribution of each member in some appropriate way can serve this purpose (Bretzke & Vassileva, 2003). Again, however, the visualization if not well-thought can be a double edged sword. If only a small portion of all users are active, it wouldn’t be a good idea to provide a whole picture of the community, because the behaviour of the inactive users can discourage the user from contributing, if she identifies with them. It is possible, and even in some cases it is recommended (Erickson, 2003) to misrepresent the actual level of contribution. We implemented visualization of the user community in the previous version of Comtella for sharing academic papers among graduate students. It showed the currently online members as stars with size dependent on the number of their shared files. The size was relative to the total number of files shared at the moment. This lead to some confusion among the users, since it was possible to share very few files and one day appear as a tiny star but the next day as the biggest star, since there was currently nobody online sharing more files. Also, when very few stars were present, the visualization actually worked discouraging, making users feel alone. The conclusion was that we needed to represent always all the members and to have absolute rather than relative classification of the contribution level, so that users see consistently their size grow with their contribution and not in relation to who is currently on line. The good news was the fact that users actually cared about why their star that was so big and bright the previous day has shrunk to a dwarf. In the Comtella version for the Ethics and IT class we had a permanent visualization of all


61

users, independently whether on- or off-line. The online users were visualized with filled circle, while the offline users were visualized with empty circle. There were 4 possible sizes and they were used to show different aspects of contribution: number of new shared papers, number of papers downloaded from others, frequency of logging into the system, etc. The visualization was well used by the students, and the questionnaire showed that students cared about how they appeared – the majority replied that if they see that they are little stars and there are many large stars, they will try to contribute more to improve their standing in the community. 3.4. Status: Combining Reward and Fear Discrete emotions have unique appraisal patterns, motivational functions and behavioural associations. According to the theory of fear, people feel fear when they perceive some threat to themselves or their properties. This fear makes incoming messages, especially those containing reassuring information or information about how to avoid the threat more persuasive (Nabi, 2002). One possible persuasion strategy based on fear will arouse fear in the user to loose something, for example, some privilege. At that point, information how to avoid the problem is provided, e.g. by contributing more resources. This information will be more persuasive than just a general advice by the system.

Figure 1: The Comtella Interface showing the membership card and explanation of the level. We implemented in the Ethics and IT version of Comtella this idea by introducing a set of hierarchical memberships in the community depending on the level of contribution of the user. The levels are currently three: “bronze”, “silver” and “gold” (see Figure 1). The users with higher level membership are rewarded with better search functionality as explained in section 3.2. and better visibility (there is a special view “By Status” in the visualization, where all the gold, silver and bronze members can be seen). Therefore, according to the reciprocation theory (Cialdini, 2001), there are rewards for higher participation and higher status. However, when a user has reached a gold membership level, there is little incentive for her to contribute more. Therefore, the status is only temporary, in our case, on a weekly basis. According to the theory of fear (Nabi, 2002) if a silver- or gold-status user, who has already enjoyed some privileges, stops contributing to the community and the system shows a warning, this will arouse fear that her membership will be degraded in the next week. At this time, a message related to the actions that the user can take to avoid demotion will provide effective persuasion. This was achieved through a window showing


62

the level of contribution of the user with respect to several factors in comparison with the top-contributor for the week for each factor. Essential for this strategy is how the participation level is calculated. In our experiment we used a weighted sum of the following factors (in decreasing order of the weight): number of new resources brought, number of ratings or comments on resources, number of downloaded resources from others, relative frequency of logging in, total time spent on line. The students were very curious to know the formula and immediately attempted a variety of ways to “cheat” the system, especially the ones that didn’t involve significant effort: keeping logged in for long time, logging in and out frequently, downloading many files from other peers. We also saw users who contributed new links that were not related to the topic of the week, and unfortunately, they were successful, since our participation formula did not take into account the quality of the resources shared. 3.5. Results In the first 6 weeks of the class we used Comtella without status, visualization and reward for participation. We introduced the version implementing the motivational mechanism in the middle of the term (after week 7). There were 35 students registered in the class, 29 of which used the system actively. Figure 4 shows the number of new links shared by students each week. The bar for week 6 represents two weeks, since the two weeks discussed the same theme. It is evident that the number of new articles shared over the last four weeks is about twice bigger than that in the first six weeks (641 versus 332). Although there is a decrement in the last two weeks (due to the fact that the students had to focus more efforts on their course projects, scheduled for presentation in the last week of the class), the number of new resources shared increased significantly. Besides, other kinds of contributions, such as ratings and comments of articles, increased as well after the motivation mechanism was introduced in the 7th week. However, the quality of shared resources somewhat decreased. Several (4) users shared a lot of articles not related directly to the topics of the class to gain a higher membership level or maintain their gold level. One extreme case was the top contributor of the class, who shared 131 of the 973 links (13 %), of which 40 (~30%) were irrelevant. Since the hierarchical memberships were introduced into the community, the quantity of the users’ comments increased but the length of the comments became shorter on average. Obviously, in the next version of the system, the participation metric has to include a measure of the quality of shared resources and comments.

Total number of new links shared per week

050

100150200250300

1 2 3 4 5 6 7 8 9 10

weeks

num

ber

of li

nks

Figure 4. The total number of new links shared by all students over the 10 weeks.

(The motivational version was introduced after week 6.) 4. Ensuring high-quality contributions (current work) Obviously, integrating persuasion mechanisms can effectively encourage participation and contributions in a P2P community. However, whenever there is some reward (independently whether it is tangible or simply social visibility), people will be tempted to cheat the system. Therefore, adequate measures should be taken to discourage


63

cheating behaviours. Our current mechanism allows users to cheat by submitting low quality resources. It is important to include controls of quality, and it would be the best if the community, i.e. the users are empowered to exercise this control. We are currently designing a mechanism to encourage users to rate fairly resources and comments. One possible approach is to include the quality of contributed resources (e.g. the average rating by other users) in the participation formula. However, this may encourage users to rate highly the papers of their friends, or generally inflate the ratings. To ensure fair ratings it will be necessary to measure in some way the credibility or competence of users as raters, e.g. to compute a reputation (Wang & Vassileva, 2003), similar to the “karma” used in Slashdot.org. Incorporating techniques measuring similarity of users’ ratings will allow forming sub-communities of users with similar interests and criteria, which can be useful to extend the system functionality into recommending resources (similar to collaborative filtering). The more ratings a user submits, the better recommendations she can expect from the system, which will create a positive feedback loop and reward for fair and frequent ratings. The visualization will be able to show users based on their reputation, and clustered by similarity of ratings, which will elevate user participation to a higher, more knowledge-dependent level. Acknowledgements: The authors thank Jeremy Long for implementing the interface of the Comtella version for sharing course materials. This research has been supported by the NSERC Discovery grant of the first author. Comtella is available for download from http://bistrica.usask.ca/madmuc/comtella.htm References Bretzke H., Vassileva J. (2003) Motivating Cooperation in Peer to Peer Networks, Proceedings User Modeling

UM03, Johnstown, PA, June 22-26, Springer Verlag LNCS 2702, 2003, 218-227. Cialdini. R. (2001) Influence: The Science of Persuasion. Scientific American, (Feb. 2001), 76-81. EDUTELLA: a P2P Networking Infrastructure based on RDF. In Proc. of 11th World Wide Web Conference

(Hawaii, USA, May 2002). Erickson, T. (2003) Designing Visualizations of Social Activity: Six Claims, In Proc. CHI’03. Kostuik K. and J.Vassileva (1999) Free Market Control for a Multi-Agent Based Peer Help Environment, in

Proceedings of the Workshop on Agents for Electronic Commerce and Managing the Internet-Enabled Supply Chain, held in association with the 3rd International Conference on Autonomous Agents (Agents '99), Seattle, May 1-5, 1999.

Nabi. R. (2002) Discrete Emotions and Persuasion, The Persuasion Handbook: Developments in Theory and Practice, In J. Dillard & M. Pfau, Sage Publications, ISBN: 0761920064, P 289-308.

Neidl, W., Wolf, B., Qu, C., Deceker, S., Sintek, M., Naeve, A., Nilsson, M., Palmr, M., AND Risch, T. (2002) Sun L., Upadrashta Y., Vassileva J. (2003) Ensuring Quality of Service in P2P File Sharing through User and

Relationship Modelling, UM03 Workshop on User and Group models for web-based adaptive collaborative environments, Johnstown, PA, June 2003.

Shirky. C. (2003) Fame vs Fortune: Micropayments and Free Content, First published September 5, 2003 on the "Networks, Economics, and Culture" mailing list, available online at: http://shirky.com/writings/fame_vs_fortune.html

Vassileva J. , J. Greer, G. McCalla, R. Deters, D. Zapata, C. Mudgal, S. Grant (1999) A Multi-Agent Approach to the Design of Peer-Help Environments, in Proceedings of AIED'99, Le Mans, France, July, 1999, 38-45.

Vassileva J., R. Deters (2001) Lessons from Deploying I-Help, in J. Whatley & M. Beers (eds.) Proceedings of the Workshop on Agents and Internet Learning, AIL'2001 at the Autonomous Agents'2001 Conference, Montreal, May 28, 2001.

Vassileva J. (2002) Supporting Peer-to-Peer User Communities, in R. Meersman, Z. Tari et al. (Eds.) "On the Move to Meaningful Internet Systems 2002: CoopIS, DOA, and ODBASE" Coordinated International Conferences Proceedings, Irvine, 29 Oct - 1 Nov. 2002, LNCS 2519, Springer Verlag: Berlin-Heidelberg, 230-247.

Vassileva, J. (to appear) Harnessing P2P Power in the Classroom, in Proc. ITS’2004, Maceio, Brazil, August 2004. Wang Y., Vassileva J. (2003) Bayesian Network-Based Trust Model, Proc. of IEEE/WIC International Conference

on Web Intelligence (WI 2003), October 13-17, 2003, Halifax, Canada.


64

A process to analyze interactions in collaborative systems with text-based computer mediated communication tools

Marcos Augusto F. Borges1, M. Cecilia C. Baranauskas2

1 Instituto de Computação – Universidade Estadual de Campinas (IC/Unicamp) Rua Barão de Paranapanema, 400, ap. 112 – 13026-010 – Campinas – SP – Brazil

[email protected]

2 Instituto de Computação – Universidade Estadual de Campinas (IC/Unicamp) Caixa Postal 6176 – 13083-970 – Campinas – SP – Brazil

[email protected]

Abstract. It is a usual goal nowadays, to improve the collaboration among agents that interact through computer systems. It is important to establish a process to analyze this kind of collaboration. This work presented a process based on an analysis framework (FAnC) and on systems to support this framework application. CollabSS system supports the collaboration analysis while the interaction is happening, being a useful tool for agents that have the role of being collaboration facilitators. CoPA system was created in order to support a more detailed “a posteriori” analysis.

Key Words. CSCW, collaboration analysis

1. Introduction It is a common goal today to improve interaction among agents in professional activities and in learning process (Menascé, 1998; Mantovani, 1996). Collaboration is a mutual effort of a group to reach a goal, having all the agents participating in all activities (Scrimshaw, 1993).

With the advent of Internet, a lot of self-defined collaborative systems were developed. Some of them use text-based computer mediated communication tools as the way agents interact. This work focused on this kind of collaborative systems.

A systematic way to study how agents interact in collaborative systems is not usual in literature. One explanation could be the difficulty to conduct this analysis, supported by systems that help the analysis in a fast and efficient way. Some proposals as the “speech acts” theory (Searle, 1969), LAP (Kethers and Schoop, 2000) and the discourse analyses (Clark and Schaefer, 1989) can find important information about the interactions, but are very time consuming. The use of these proposals can almost be impossible in interactions having a great number of messages.

Collaborative activities can reach better results when an agent acts as a “collaboration facilitator”. This facilitator must try to keep the interaction in a collaborative atmosphere. In order to do that, the facilitator shall analyze the interaction during its course, trying to understand how the agents are collaborating. The analysis of interactions, especially the synchronous ones, that have many agents, could not be possible without a support tool to the facilitator. This kind of tool was not found in literature.


65

mailto:[email protected]

This work presents a process to analyze interactions. This process can support not only interaction analysis, but also the facilitator during the course of the interaction. The process proposed is based on FAnC framework and CollabSS and CoPA systems. The framework FAnC was developed to analyze interactions. Both systems aims to support FAnC. Collabs supports facilitator during the interactions and CoPA supports a more detailed post-analysis.

2. The FAnC framework for interactions analysis The term “speech” is used in the context of this work to denominate each individual message sent by a user or by the system itself as part of a conversation. These speeches are classified identifying the speaker, the listener and the class.

The identification of the senders and receivers are necessary to analyze the different roles the coordinator and the other players have in the situation. The type of interaction and needs are supposedly distinct for these two different categories of users. A speech could have been sent to all players, but its context could indicate that the sender directed it specifically to someone in the group. This type of speech was classified in CA as a directed message.

Guerrero et al (2000) proposed a classification for the type of information conveyed by speech in a collaborative scenario. Inspired by their findings, we proposed that speeches should be classified according to the kind of information they transmit.

In the context of this framework, we define a “conversation” as a sequence of related speeches. The conversations are then classified according to the user who initiated it (identifying if s/he is in the role of a facilitator or not), the class of most speeches and the size of the conversation (the number of speech it contains). We can also identify conversations occurring at the same time (in parallel) and those that subdivide in branches that continue in parallel.

Cláudia to ALL: Should we increase the machines capacities? Cecilia to ALL: I believe the capacities are enough. Heloísa to Cecília: Did you transfer the raw material? Cláudia to Cecília: Isn’t the low capacity the problem? Cecília to Heloísa: I forgot! Heloísa to Cecília: So, transfer now! Cecília to Heloísa: Ok. Transfered. Cláudia to ALL: Why don’t we reach the goal, if the capacity is enough? Heloísa to Cláudia: Maybe the transfer size is the problem.

Figure 1. A conversation graphic (top) and the interaction text (below). C1 to C4 are classes and 1 and 2 interaction conversations.


66

In the proposed framework, each speech in a chat log file is analyzed and its class identified. The conversations are delimited. After that, the chat interaction during an entire session is represented graphically in a conversation graphic. This graphic represents users as horizontal lines and speeches as diagonal lines connecting the horizontal ones; each line connects the sender to the receiver horizontal line (from left to right). When the message has more than one addressee, this will be represented by lines having the same origin (the left side of the line). Lines are labeled with the speech classification. A square marked with an “x” identifies the receiver of each speech and a numbered square the sender. A bold number is used in the beginning of the conversations. The time line increases from left to right.

3. CollabSS and CoPA Based on FAnC, we designed an environment composed by two systems (CollabSS and CoPA). The architecture for this environment is represented in the Figure 2. The environment design had the goal of making easy the integration with any external CMC environment, in which interaction occurs through text based tools.

CollabSS/CoPA Environment

CollabSS System

Settings

CoPA System

Settings

(Dis)Connection events

Speeches & Conversations

Conversation starters

Agents

Interaction data

External CMC

environment

Speeches classification

Figure 2. CollabSS/CoPA Environment.

CollabSS (Collaboration Support System) (Borges & Baranauskas, 2003B) is the system which interconnects with the external system and saves the interaction data. When the external system uses a collaboration facilitator and the interaction is synchronous, CollabSS support the facilitator in his/her goals of supporting the collaboration by presenting hints and information concerning participation and interaction of the participants during the activities (for example, pointing when a user is not interacting for a long time, presenting barr graphics with the number of messages each user sent to the interaction, etc.). CollabSS presents also FAnC proposed graphics representing the interaction taking place at the collaborative tool, to help the facilitator to evaluate it. CollabSS gets data previously analyzed in CoPA to use them in the facilitator support.

The process of making a detailed a posteriori analysis of the interaction is important to evaluate how the collaboration is occurring through the tool, but it is very time consuming. CoPA (Collaboration Post Analysis) (Borges and Baranauskas, 2003D) is a tool to support this analysis, after the learning activities. CoPA provides support for evaluation and classification of each speech and conversation, to capture information about the types of interaction and collaboration present in it. CoPA saves data to be used by CollabSS in the course of interactions. CoPA can be used with any speech classification respecting the FAnC restrictions. Borges and Baranauskas (2003A) present a typical analysis that can be based on CoPA.


67

CollabSS has an agent to trace the interaction and to analyze the data detecting moments when the interaction in the chat is not satisfactory, i.e. the density of speeches by time is low, or when some user is not participating in the discussion: we call these moments as “collaboration support moments”. A collaboration support moment is a special moment when the facilitator must act to instigate collaboration. When a collaboration support moment is detected, CollabSS shows to the facilitator a button with a lamp in the main window. Clicking on this button, the system will present the “Sending a Conversation Starter” window. This window presents a conversation starter. A conversation starter is a special speech that usually starts collaborative interactions among agents. The facilitator can change the message, ignore it, send it to all agents of the interaction or, when the moment is based on the interaction of a specific agent, send it to him/her.

CollabSS also provides on line dynamic representations of interaction. During an interaction, these representations intend to make easier to facilitators the analysis of the course of the conversations, the way agents are interacting with each other and how many speeches are appearing in the interaction by user, by conversation and by time. The facilitator can make use of the conversation graphic and the condensed conversation graphic (Figure 3) and a bar chart with the number of speeches by user.

Using the conversation graphic (Figure 3) the facilitator can analyze the user’s participation, the speech density in a short interval of time and the pattern of conversations. The horizontal lines represent the agents. During the course of a conversation, the facilitator can select the agents s/he wants to see represented in the graphic. The time goes from left to right: in figure 3 the extreme right is the time at the moment the representation is presented and it goes to the left till 25 seconds before. The graphic has a horizontal scroll, to enable an analysis of the whole course of the interaction, since its first speech. Each speech is presented as a diagonal line, having the sender at the left side and the receiver at the right side.

Using the condensed conversation graphic (Figure 3), the facilitator is able to analyze the agents’ participation and the density of speech in a larger interval of time. The use of colors represents the amount of speech sent by the user represented at the line in the time interval (speech density). The time scale can be configured to represent a bigger or a smaller time interval.

CoPA receives as input (data that can be direct generated when using CollabSS): speeches, the agents enrolled and the connection and disconnection events. CoPA also gets: conversation starters previously select at CoPA and the speeches classification.

Based on input data, CoPA makes available tools to analyze speeches, to define conversations and to evaluate interaction. Figure 4 shows a snapshot of CoPA main window. From this window, a user can:

• access the “Conversation Construction” tool, where the user can select a speech and define: which speech precedes it; the class of it;

• access the “Conversation Starters’ Edition” tool, that offers an edition tool for conversation starters (including insertion and deletion). To define a speech as a conversation starter, the user must evaluate the type and the size of the conversation this speech starts: a conversation starter should have begun a long conversation of some special classes interesting to the type of interaction;

• evaluate the connection/disconnection events;

• call bar charts with quantitative analysis of conversations;

• ask for, to all interaction or to a specific conversation:

- basic information;

- conversation graphic and conversation graphic condensed as proposed by FAnC;

- bar charts with quantitative analysis of speeches.


68

Figure 3. Conversation Graphic and Conversation Graphic Condensed

Figure 4. CoPA system main window

4. Conclusion This work presents a process to analyze interactions in collaborative systems, composed by the FAnC framework and CollabSS and CoPA systems.

FAnC makes possible to conduct analyzes that can find important information about the interaction, without needing a long time to conduct it. FAnC has as unique characteristics:

• it defines important information of the interactions;

• it proposes easy access graphical representations;

• it presents a set of heuristics to support facilitator during and after an interaction.

FAnC was used in experimental analysis and reach the proposed goals. CollabSS was experimented also in a participatory heuristic evaluation (Borges and Baranauskas, 2003C), reaching its goals of supporting a facilitator during the course of interaction. At last, CoPA was used to analyze interactions and it could be


69

detected that this system makes the FAnC based analysis process more efficient and precise, decreasing the time needed and minimizing the errors.

References Borges, M.A.F., Baranauskas, M.C.C. (2003A) “Supporting the Facilitator in a Collaborative Learning

Environment”, International Journal of Continuing Engineering Education and Life-Long Learning, UNESCO/Inderscience Enterprises, Genebra, Swiss, v. 13, n. 1/2, p.39-56.

Borges, M.A.F., Baranauskas, M.C.C. (2003B) “CollabSS: a Tool to Help the Facilitator in Promoting Collaboration among Learners”, Educational Technology & Society, v.6, n.1, http://ifets.ieee.org/periodical/vol_1_2003/borges.pdf (last access 18/03/2003).

Borges, M.A.F., Baranauskas, M.C.C. (2003C) “A participatory heuristic evaluation of CollabSS (Collaboration Support System)”, Proceedings of 11th International PEG Conference, CD-ROM, San Petersburg, Russia.

Borges, M.A.F., Baranauskas, M.C.C. (2003D) “The CoPA (Collaboration Post Analysis) System”, Proceedings of 11th International PEG Conference, CD-ROM, San Petersburg, Russia.

Clark, H.H., Schaefer, E.F. (1989) “Contributing to discourse”, Cognitive science, n.13, p. 259-294.

Guerrero, L.A. , Alarcon, R., Collazos, C., Pino, J.A., Fuller, D.A. (2000) “Evaluating Cooperation in Group Work”, The Sixth International Workshop on Groupware, Workshop Proceedings, IEEE Computer Society, Madeira, Portugal, 28- 35.

Kethers, S., Schoop, M. (2000) “Reassessment of the action workflow approach: empirical results”, Fifth International Workshop on the Language-Action Perspective on Communication Modelling (LAP 2000), Aachen, German, p. 151-169.

Mantovani, G. (1996) “Social Context in HCI: a New Framework for Mental Models, Cooperation, and Communication”, Cognitive Science, v.20, p. 237-269.

Menascé, D. (1998) “Educational challenges and opportunities in the web era”, Anais do XVIII Congresso da Sociedade Brasileira de Computação, 1, Belo Horizonte-Brazil, p. 433-444.

Scrimshaw, P. (1993) “Cooperative writing with computers”, Language, classrooms & computers, Edited by P. Scrimshaw, Routledge, London-UK, p.100-110.

Searle, J.R. (1969), Speech Acts: An Essay in the Philosophy of Language, Cambridge University Press.


70

Towards interaction modelling of asynchronous collaborative model-based learning

E. J. R. de Castro1, G. M. da Nóbrega1, E. Ferneda1, S. A. Cerri2, F. Lima1

1 Universidade Católica de Brasília, Brasília, Brazil

2 Laboratoire d’Informatique, de Robotique et de Microélectronique de Montpellier, Montpellier, France

[email protected], [email protected], [email protected], [email protected], [email protected]

Abstract. In the last decade, the design of collaborative discovery learning environments (CDLE’s) has received increasing attention. Such a design perspective brings up to the educational context the possibility of exploiting the so-called Model-Based Reasoning approach, by providing to learners the opportunity of collaborating while building models to represent observations. In this paper, we are concerned with the design of CDLE’s in which building models instantiates as a theory formation process. Such a process is provided as a synergetic combination of both inductive and hypothetical-deductive approaches. The “moving engine” allowing a theory to evolve is the notion of contradiction: learning is supposed to occur as a side effect of contradiction detection and overcoming during theory formation by learners guided by a coordinator. Inspired in recent relevant work concerning Computer Supported Collaborative Learning (CSCL) architectures, we propose an architecture to support the process of asynchronous theory formation, allowing a student both to work individually and to contribute to the group discussion. Out of the proposed architecture, we draw-up related questions that would address supporting a coordinator to guide discussions on the basis of group interaction analysis. Keywords. Model-based learning, CSCL architectures, asynchronous communication, supported coordination.

1 Introduction

In Educational literature, Discovery Learning appears as an approach in which the learner builds up his/her own knowledge by performing experiments within a domain and inferring/increasing rules as a result. Such an approach “[...] has appeared numerous times throughout history as a part of the educational philosophy of many great philosophers particularly Rousseau, Pestalozzi and Dewey, ‘there is an intimate and necessary relation between the process of actual experience and education [1]’. It also enjoys the support of learning theorists/psychologists Piaget, Bruner, and Papert, ‘Insofar as possible, a method of instruction should have the objective of leading the child to discover for himself’ [2]” [3]. Such a constructivist approach has been largely exploited for the design of computational artefacts with learning purposes, the so-called Discovery Learning Environments (DLEs). One known feature of such environments is the autonomy degree required for students to succeed while handling a domain.

In his introduction [4] to the book “Collaborative learning: cognitive and computational approaches” P. Dillenbourg considers the notion of collaborative learning in a three-dimension space generated by the following three axis: (i) the scale of the collaborative situation in terms of the amount of people involved, (ii) what is actually concerned to learning, and (iii) how collaboration is provided (face-to-face or computer-mediated, synchronous or not, etc.).

In the last years, several scholars have been investing efforts to bring together both collaborative learning and discovery learning, thus leading to the emergence of the collaborative discovery learning approach [5]. In order to show the effectiveness of the collaborative discovery learning approach, a number of systems have been designed, such as Belvedere (groupware for learning scientific argumentation) [6] and GARP [7].


71






Current efforts on CSCL include the design of computational models of collaborative learning interaction such as to improve support and guidance to humans taking part in the process. In such a context, relevant work has been invested in asynchronous interaction analysis, e.g. [8], however, the major focus is often on free-speech discussion. On the other hand, whenever communication is structured (model-based), efforts have often been invested either on synchronous interaction or the charge of coordination is yet considerable for participants (e.g. CMapTools).

In this paper we are concerned with the challenge of approaching interaction analysis by considering, as a first step, an environment for asynchronously collaborating by building models. Based on a forum structure, our conjecture is that such an environment facilitates perception by the group of its work as such. In §2, we propose an architecture for collaborative learning environments to support the asynchronous construction of models by a students group. Then in §3, interactions within such an environment are illustrated through a scenario and a discussion is opened on how the proposed architecture would facilitate the design of artificial agents capable of positively influencing collective model construction. Finally, in §4 we present our concluding remarks and point out both ongoing and further work.

2 On asynchronous collaborative model-based learning: an architecture

In [9], the author describes several possibilities of architectures for CSCL environments. These architectures are presented upon the design pattern known as Model-View-Controller (MVC): “Model is an internal representation of a semantic model of the problem of interest. The View displays the model in some visual representation”, and a “[…] Controller enables the user or the environment to modify the state of the Model”.

Inspired in Suther’s discussion, we draw-up our architecture on the basis of the MVC design pattern. Figure 1 shows our architecture named AC-Hybrid, where “A” stands for Asynchronous and “C” for Collaborative. We recall those features further below. In Figure 1, Client A and Client B are each representing a participant’s machine (abstractly speaking). Server is representing a machine controlled by the group’s Coordinator. In a general manner, a Model is the result of representing an observation. We distinguish three kinds of Models, namely Individual Model, Global Model, and Collaborative Model. By handling an Individual Model, a student has the opportunity to organize his/her ideas in a private manner, before he/she feels able to propose them to the group. Such a Model lives in a Client and is both viewed and controlled by the participant owning it. Individual Models may be replicated in Client machines in order to account for memory versioning, but a single version is modifiable at a time (the more recent one).

Modelo Individual Model

Global Model

Client A View Control

Modelo

Client BServer

Modelo

Coordinator

Global Model

GlobalModel

CollaborativeModel

IndividualModel

View Control View Control

Figure 1: AC-Hybrid Architecture.

The Global Model should represent group consensus at a given moment. On the one hand, it should be stable such as to be usefully exploitable for the group’s further elaborations. In other words, it would serve as a current group memory available anytime to be inspected by group members. On the other hand, a Global Model is supposed to continuously evolve such as to capture the group’s cognitive progress. Similarly to Individual Models, Global Models may also be versioned such as to keep track of group evolution. These two features assigned to Global Models – stability and predisposition to evolve – have suggested us the need for an additional Model justifying thus the “AC” part of our AC-Hybrid architecture: the Collaborative Model.

The Collaborative Model plays the role of an intermediate Model candidate to replace the Global one. It arises as a suggestion from a group member who aims at modifying the Global Model. Such a suggestion would then be


72

submitted to the group’s analysis, and thus, it would trigger a debate. The environment in which the debate takes place borrows the main structure from a forum. While the idea of asynchronous communication seems convenient here to hold a discussion, our objective is however to provide a group with a mean to reach a consensus, a kind of drawn-up conclusion about one’s suggestion. The idea of using Models rather than free-speech usually considered within forum tools appears here to respond to that objective, thanks to a Model’s underlying structure. It is up to the Coordinator to decide when to stop a debate and to replace the Global Model. Careful analysis of interactions is crucial to allow the Coordinator both to guide the debate and to decide about Model replacement.

We see then the MVC design pattern as a suitable one for the purposes of the above depicted architecture, since several models may exist in a given moment (m Individual, 1 Global, n Collaborative), each one viewed/controlled by eventually different users, according to the intended dynamics.

3 An illustrating scenario

In this section we develop a scenario aiming to illustrate some possible interactions between participants supported by a system based upon the proposed AC-Hybrid architecture. These interactions would allow the evolution of the group’s Global Model through a discussion relying on Collaborative Models. For such, we consider the following: • Models constructed by participants are considered here to be logical theories; • A classical toy domain widely exploited by scholars on Artificial Intelligence; • A conceptual model - called Phi-calculus - originally proposed to support human-computer collaboration

during theory formation [10]. Phi-calculus was then instantiated into the context of Human Learning, grounding the design of a Web-served Learning Environment which was submitted to real learners in Law1 [11]. The theory formation process underlying the model is supposed to promote learning as a side-effect. Our first elaborations aiming at extending Phi-calculus to a human-human collaborative educational context are reported in [12].

Phi-calculus relies on a synergetic combination of both inductivist and hypothetical-deductive rationales. The “moving engine” of the theory formation process is the notion of contradiction [13]: a theory is supposed to evolve by contradiction detection and overcoming. Contradiction should arise during confrontation between current theory and incoming experiment (Examples/Counter-examples). It is supposed to reveal disagreement between individual’s observations and the current available theory. Recent relevant work on collaborative environments has confirmed the interest of socio-cognitive conflict theory to learning [14].

During the theory formation process, communication between a human agent and his/her Artificial Agent takes place by means of constrained dialogues [15]. These correspond to messages formalized under the form of the speech acts Ask and Tell [16], representing, respectively, (i) agent A asks something to agent B (or vice-versa) and (ii) agent A informs something to B (or vice-versa). Also, messages in constrained dialogues rely on exchanging (asking and telling) what we have called knowledge types within Phi-calculus. Such types are not exploited here: they should account to the states that an evolving theory (or a concept being formed) should assume.

Recalling the AC-Hybrid architecture from §3, interactions between a student and his/her own machine to modify his/her Individual Model are based upon original Phi-calculus model of human-computer collaboration. It is important to notice that within our current implementations, the system interface renders the above mentioned formalism totally transparent to the user-learner. In addition, the student is provided with a machine support to the theory construction process: an induction engine [17], accounting for Learning from Examples approach.

Before depicting our scenario let us state that one can contribute to a debate by either questioning a Model or proposing a solution by suggesting a candidate Model to replace the Global one. In both cases, the contribution appears as a Model along with a justification for the question or solution. The speech acts Ask and Tell are used to represent, respectively, the questioning situation and the proposition one.

1 Postgraduate students from Université Montpellier II, Montpellier (France).


73

Let us now start our hypothetical scenario by supposing that a History class interested in the study of historical monuments intends to formalize the concept of “Arch”. We begin by considering that a Global Model is available: it has been previously constructed by the group out of some images supplied by the Coordinator. The annotation of each image fits on the Model. The whole situation is illustrated in Figure 2.

ModelIndividualModel

GeneralModel

Machine B

View Control

Model

Machine CMachine A

Model

Coordinator

GeneralModel

GeneralModel

ColaborativeModel

IndividualModel


rectangle blocktrianglesquare

Block1 Block2

oval

stable unstable

form arch Ex1 Ex2

Ex3 C-Ex1

Figure 2: Global Model, version 1 (In order to hold the structured object annotations, concepts and their

relations to each other are built up).


Now let us suppose that the Coordinator opensup a debate by proposing an object thatcontradicts the current theory (represented bythe available Global Model). In Figure 3, ascreenshot illustrates how a pair student-agentcould detect contradiction after havingimported the Global Model to work it outindividually, and then annotating the proposedobject by clicking on a grid. Let us now suppose that, after finding asolution to overcome contradiction, the studentsuggests a candidate Collaborative Model toreplace the Global Model. The (annotation ofthe) object supplied by the Coordinator justifiesthe incoming suggestion. The situation isillustrated in Figure 4.

Figure 3: Individual Model (imported from Global Model): a student annotating the object and then system

detecting contradiction.

ModelIndividual Model

General Model

Machine B

View Control

Model

Machine CMachine A

Model

Coordinator

General Model

General Model

Colaborative Model

Individual Model


rectangle block triangle square

Block1 Block2

oval

stable unstable

form arch C-Ex2

Figure 4: Collaborative Model: suggestion from one student (relation Block2 →Arch is deleted to overcome

contradiction provoked by the - annotation of the - proposed object).

74

Another student then points out a problem with his/her colleague’s suggestion by stating that such a theory is unable to distinguish two of the previous objects. The situation is illustrated in Figure 5.

ModelIndividual Model

General Model

Machine B

View Control

Model

Machine CMachine A

Model

Coordinator

General Model

General Model

Colaborative Model

IndividualModel


rectangle block triangle square

Block1 Block2

oval

stable unstable

form arch Ex1

C-Ex2

? Figure 5: Collaborative Model: question from another student (why the Model cannot distinguish the two

objects?).

The scenario might be continued by considering a suggestion to solve the problem identified by the above student2.

From the above architecture and its illustration the following questions might deserve deeper discussion. From the learner’s perspective, how could one get some assistance to find out the actual contribution of his/her Model to the discussion? Thanks to a forum structure being exploited here, various Collaborative Models may arise as a result of propositions by different students. When proposing a Model, one possibility would be to dispose of an artificial agent, which would be able to check out for eventual conflicts with respect to existing candidates.

Concerning the Coordinator’s perspective, what would be the right moment for him/her to decide to update the Global Model? Should it mean that a consensus has been reached? In addition, it may happen that several candidate Models exist intending to replace the current Global Model; in such a case, which one to choose? One possibility would be to assume that Models deeper placed in the forum structure represent those more warmly discussed. On the other hand, even when choosing a Collaborative Model to replace the Global Model, the current discussion may yet be kept open (promising candidate Models do not necessarily need to be forgotten).

Finally, we believe that the forum structure along with model-based reasoning might facilitate qualitative interaction analysis, often more suitable and hardly accounted than quantitative one. Such an assumption lies on the basis of the fact that proposing a Model should reflect student’s active participation in a discussion, as constructivist theories claim.

4 Conclusion

In the paper we propose an architecture for CSCL environments particularly concerned with the asynchronous collaborative construction of models by students. Even if aware of the early stage of our current research, we suspect it to be promising on the basis of both previous work and the conjecture that structured model-based discussion facilitates automated interaction analysis (in spite of its pedagogical value).

We are currently working to achieve an architecture mature enough to allow us to invest on its implementation, and then to submit it to real educational experiences, as we previously did within a human-computer collaborative context. Further work includes investing on interaction analysis such as to provide and to assess coordination guidance.

2 One possibility is to reformulate the vocabulary in order to capture the distance between the two base blocks, then to find out new relations among the terms.


75

REFERENCES [1] J. Dewey, Experience and Education, MacMillan, New York, 1938. [2] J. S. Bruner, On Knowing: Essays for the Left Hand, Harvard University Press, Cambridge, Mass, 1967. [3] J. Bartasis and D. Palumbo. Theory and technology: Design consideration for hypermedia/discovery learning environments. available online: http://129.7.160.115/INST5931/Discovery_Learning.html, 1995. [4] P. Dillenbourg, ‘What do you mean by ‘collaborative learning’?’, in Collaborative Learning: Cognitive and Computational Approaches, ed., P. Dillenbourg, Advances in learning and instruction, chapter 1 (Introduction), 1–19, Elsevier, Oxford, (1999). [5] W. R. van Joolingen, ‘Designing for collaborative discovery learning’, in Intelligent Tutoring Systems, 5th International Conference, ITS 2000, Montréal, Canada, June 19-23, 2000, eds., G. Gauthier, C. Frasson, and K. VanLehn, volume 1839 of LNCS, pp. 202–211, Berlin Heidelberg, (2000). Springer-Verlag. [6] D. Suthers, A. Weiner, J. Connelly, and M. Paolucci, ‘Bevedere: Engaging students in critical discussion of science and public policy issues’, in 7th World Conference on Artificial Intelligence in Education - AIED’95, August 16-19, Washington, DC, pp. 266–273, (1995). [7] P. Salles and B. Bredeweg. A case study of collaborative modelling: building qualitative models in ecology, in Artificial Intelligence in Education: Shaping the Future of Learning through Intelligent Technologies, eds., U. Hoppe, F. Verdejo, and J. Kay, pp. 245-252, IOS-Press/Ohmsha, Japan, Osaka, (2003). [8] J. van der Pol, ‘Identifying and modeling variables in complex CSCL-situations Case study: the use of asynchronous electronic discussions’, in CSCL 2002 Workshop on Designing Computational Models of Collaborative Learning Interaction, pp.28-34, Boulder (USA), 2002. [9] D. D. Suthers, ‘Architetures for computer supported collaborative learning’, in IEEE International Conference on Advanced Learning Technologies - ICALT’2001, eds., T. Okamoto, R. Hartley, Kinshuk, and J. P. Klus, pp. 25–28, Madison, Wisconsin (USA), (August 6-8 2001). IEEE Computer Society. [10] G. M. da Nóbrega, P. Malbos, and J. Sallantin, ‘Modeling through human-computer interactions and mathematical discourse’, in Logical and computational aspects of model-based reasoning, eds., L. Magnani, N. J. Nersessian, and C. Pizzi, volume 25 of Applied Logic Series, 293–311, Kluwer Academic Publishers, Dordrecht, (2002). [11] G. M. da Nóbrega, S. A. Cerri, and J. Sallantin, ‘On the social rational mirror: learning e-commerce in a web-served learning environment’, in Intelligent Tutoring Systems 6th International Conference - ITS 2002 - Biarritz (France) and San Sebastian (Spain), june 2-7, eds., S. A. Cerri, G. Gouardères, and F. Paraguaçu, LNCS 2363, pp. 41–50, Berlin Heidelberg, (2002). Springer-Verlag. [12] G. M. da Nóbrega, E. J. R. de Castro, E. Ferneda, S. A. Cerri, and F. Lima, ‘Machine learning at the learner´s hand: a support to theory formation in collaborative discovery learning environments’, in ECAI 2004 Workshop on Artificial Intelligence in Computer Supported Collaborative Learning (to appear). [13] G. M. da Nóbrega, S. A. Cerri, and J. Sallantin, ‘A contradiction-driven approach of learning in discovery learning environments’, in XIV Simpósio Brasileiro de Informática na Educação – SBIE 2003, eds., F. F. Sampaio, C. L. R. da Motta, and M. F. Elia, pp. 453–462, Rio de Janeiro – RJ, (12–14 novembro 2003). NCE/UFRJ. (Best paper award). [14] M. de los A. Constantino-González and D. D. Suthers. A coached collaborative learning environment for entity-relationship modelling. In C. Gauthier, C. Fasson, and K. VanLehn, editors, Intelligent Tutoring Systems 5th International Conference – ITS 2000, pp. 325-333, Berlin Heidelberg, 2000. Springer-Verlag. [15] Collaborative Dialogue Technologies in Distance Learning, eds., M. F. Verdejo and S. A. Cerri, volume 133 of ASI Series F: Computers and Systems Sciences, Springer-Verlag, Berlin Heidelberg, 1994. [16] J. Searle, Speech Acts, Cambridge University Press, Cambridge, 1969. [17] M. Liquière and J. Sallantin, ‘Structural machine learning with gallois lattice and graphs’, in 5th International Conference on Machine Learning (ICML’98), pp. 305–313, Madison, Wisconsin (USA), (1998).


76



Using Patterns to Reveal E-Mail Communication Structures

Katrin Gaßner

Fraunhofer Institute for Software and Systems Engineering (ISST) Emil-Figge-Str.91, 44227 Dortmund, Germany

[email protected]

Abstract. This paper argues for the use of patterns to analyze communication structures. Patterns aggregate meaningful coincidences concerning aspects either at a certain point of time or with respect to reiterations. Subsequently, a particular kind of diagram is introduced to describe various aspects of communication, and to make explicit the actual selection of aspects that need to be considered. However, it is not attempted to define a meta-model of communication, but rather to provide different views on communication, as a means to analyze and reflect upon assumptions and propositions regarding communication structures within working groups. Further, communication patterns are used to shape hypotheses regarding the relation of cooperation and workload. In order to test these hypotheses, a prototype tool called CommunicAID:CommAn1, has been developed that enables the analysis of E-Mail communication based on patterns as proposed in this paper.

Keywords. Communication, Analysis, Cooperation, Pattern, Group, E-Mail

Introduction The term communication structure is commonly used in an intuitive manner, i.e. without a precise definition. Sometimes the term is used for official directives, guidelines, or standard procedures in organizations. In other respects communication structures vaguely denote emergent networks of communication. In this paper communication structures denote these informal virtual networks which hold important information about people’s expertise, their responsibilities and leadership for tasks beyond official competences, and about communities of practice (Wenger, 1998).

We are using E-Mails as data base for an analysis approach. E-Mail is a highly relevant medium for cooperation and communication in networked communities that can be for example private, social or organizational networks. Wellman (2000) discusses the importance of E-Mail communication for cooperation and knowledge exchange. He stresses the aspect that such media are the only means to get information in a sprawling organization where changes occur on a day-to-day basis and people are networked beyond the organizational structures. Especially for informal communities E-Mail logs offer the option to get information about group relations. Such communities of practice (Wenger, 1998) grow in addition to formal structures, hierarchies and leadership. They are often informal learning groups. Consequently, Gloor et al. (2003) conclude with several fields of application where it is important to know something about the existence of such communities: Such communication reflects innovations that are carried out, can help to identify people who are the knowledge sources. Based on the E-Mail analysis communication processes could be streamlined and they could be a basis for awarding people. We also want to point on the importance to detect experts in order to match people to tasks, to groups or only to integrate expertise into decisions. This is important for both organizational groups and learning groups.

This paper investigates whether certain aspects of cooperation are reflected within (E-Mail) communication structures, for example work load, phases of learning processes, running teams, conflicts, roles of people or

1 At the Fraunhofer Institute for Software and System Technology ISST CommunicAID labels a group of tools used for communication support and analysis. CommAn stands for Communication Analysis.


77

personality traits. Approaches as presented by Tyler et al. (2003) and by Gloor et al. (2003) also use E-Mail logs for the analysis of community structures. They synthesize graphs based on information about senders and receivers where edges represent communicative relations and vertices stand for persons. Based on the ‘betweenness’ criterion sub graphs are detected that represent communities and even show leadership. For these approaches short paths analysis-a graph algorithm-is utilized in order to separate sub graphs by deleting edges.

Both approaches, Tyler et al. (2003) and Gloor et al. (2003), reduce communicative relations between people to single unweighted and undirected edges. In our approach we integrate both aspects – direction of communication and frequency - because we assume both as relevant aspects for example for coordinative contributions and delegations which are roles in communication process. Communication structures are characterized by the quantity and frequency of communication and by the kind or rather fashion of communication (cf. Lewandowski, 1994). Different from Gloor et al. (2003) we do not focus on knowledge communication and communication threads related to a superior task. When analyzing E-Mail communication randomly in an organization, this assumption does not apply.

In a wider context this paper fits to approaches that analyze traces of cooperation that manifest as utterances in computer mediated communication. In his theory of groups McGrath (1991) states that an action can only be understood within a situation as a pattern or sequence of acts: “Instead of using an act-by-act categorization of ongoing process, group interaction can be studied by constructing what might be regarded as ‘qualitative aggregations’ of acts” (McGrath, 1991, p. 169). For analysis these acts have to be aggregated “over types, over members, or over periods of time” (McGrath, 1991, p. 168). Gaßner et al. (2003) present a classification of analysis methods for collaborative activities. There the raw data for analysis is separated into states and actions that could be analyzed under two perspectives: summary or structural. This approach of E-Mail analysis takes up the idea of the aggregation of activities - in this context communicative acts – in order to find communication structures meaningful for cooperation.

Situations and Time Intervals to Describe Communication Structures as Patterns Communication is a continuous process that can be described under different aspects (e.g. topic, media, people). Figure 1a shows an example of a representation suggested in this paper as means to make aspects explicit. Horizontally, time intervals show consistent phases concerning a given criterion annotated at the vertical dimension. Vertical dotted lines at a point in time define situations that refer to the set of criteria combined in the diagram. Within this representation a communication structure is a pattern in the horizontal and/or vertical dimension: a communication structure means a coincidence of characteristic values of aspects or a regular reiteration of situations. This is similar to state and process patterns as mentioned in Gaßner et al. (2003) but not only related to synchronous work. In the following examples, the diagrams are produced manually. They show principles of patterns. In order to detect real communication structures, additional methods have to be used to recognize the values of a given set of aspects in logs of computer mediated communication. The suggested representation could be seen as an epistemic form (Collins & Ferguson, 1993; Gaßner, 2003) used in this case to challenge intuitive assumptions of communication.

time

numberof

people

topic

t1 t2 t3 t4

Count: 3 Count: 2 Count: 4

Topic: 1 Topic: 2 Topic: 1

addressee

speaker

P2

P3

P1

P1 P3

P1

Scenario

face-to-face time

a) b) Interest group c) Interaction

(receivers)

topic

sender

E-Mail

Medium

Topic1

Py

Pg1,..., Pgn

Px

Topic1

Ph1,..., Phn

time

Figure 1. Examples for patterns of communication processes.

Applying this approach to E-Mail communication means to use a set of E-Mails as basic data for the analysis and to find relevant aspects to describe these communicative acts in order to find meaningful communication patterns. As mentioned in the Hewlett Packard study (Tyler et al. 2003), ‘leadership’ is one possible example for such a meaningful pattern. Figure 1b shows a possible definition for an interest group that takes the topic as one


78

defining aspect. The working thesis could be that all people who contributed to this topic at least once are part of the interest group. In the given example, persons Px and Py both contributed to topic1 and would therefore be part of the same interest group. Taking this hypothesis it can be tested empirically if this is subjectively true for people who are recognized as member of an interest group. Also suggested in Figure 1b are receivers. For them it is not sufficient to claim that a person who got an E-Mail for a topic also belongs to the concerning interest group. This would imply that receivers of advertising E-Mails are always interested in that. Hence, the receiver aspect should be deleted from the diagram. Figure 1c shows an example for an interaction that might denote a communication problem. In a face-to-face situation a person 1 addresses the contribution to another person 3. In the next step person 2 contributes but not person 3. Person 2 addresses again person 1 who again addresses person 3 within the contribution. This could mean that person 1 ignores person 2. If this interaction is iterant for these people this can be interpreted as a social problem among this group of people. If it is only iterant that person 2 is talking without getting the turn it might be concluded a special character for person 2 as egotism or craving for recognition.

The Tool CommunicAID:CommAn CommunicAID:CommAn has been developed as an initial prototype that supports the previously mentioned analysis of E-Mail communication processes. It consists mainly of two main functionalities the visualization of structures and filters that define which information is to be shown within the diagram. The visualized structures are directed or undirected graphs. Colored edges (in the following reduced to gray tones) map relations between persons or entities into ranges of numbers or intensities.

View Aspects Visualization a) Message processing.

receiver P1 and processing time

sender and contribution

C1(P1)Medium

E-Mail

receiver P2 and processing time

tP1(C1)

C2(P1) C3(P1)

tP1(C2) tP1(C3)

tP2(C1) tP2(C2) tP2(C3)

b) Number of E-Mails to

addressees by one sender.

addressee

P3 P2, P3

sending a contributionby person P1P1

P2, P4

P1

Medium

E-Mail

P1

P2, P4

c) Number of E-Mails to addressees by several senders.

addressee

P3 P2, P3

contributionby person

P2P1

P2, P1

P3

Medium

E-Mail

P1

P1, P4

t1t0

P3

P1

Figure 2. Summary analyses of E-Mail communication.

Figure 2 shows some examples for patterns meaningful for communication processes. Presumably, it is not possibly to deduce patterns but rather to proof patterns that are hypothesized preliminary. Therefore, we mention some hypotheses concerning these patterns. The idea is, to check whether the E-Mails mirror these propositions and to carry out interviews with users afterwards in order to test these hypotheses. The visualization methods in CommunicAID:CommAn are used to condense data in a manner that people can interpret more easily. The diagrams mentioned before are used to describe the principle but are not appropriate to represent large amounts of data.


79

Message processing Figure 2a shows one example where the amount of information is counted for further interpretation. Three parameters are mentioned as relevant parameters that influence the processing time: the number of bytes of the E-Mail itself, the length of attachments, and the number of linked URLs. For a concrete E-Mail values for these parameters have to be combined by an additional heuristic factor. The aspect’s diagram relates the contributing person to the receivers together with a time interval for processing the E-Mail. To improve clearness, for each receiver one time axis is inserted. The graph at the right shows a possible visualization of a set of E-Mails. In this diagram the gray tones symbolize grades of work load where the peak of the arrow points to the sender of an E-Mail and the wide end of the edge points to the receiver of the E-Mail: The darker the gray tone the bigger the workload.

Hypothesis1: So far this summary analysis can be related to a task the visualization says something about the relation between the expected work load and the observed work amount, hence, the degree of difficulty or even motivation. Both is possible getting an impression about the work load of a group and the work load of individuals. Especially in distributed learning scenarios where only one main task is to be solved, such summary analysis can be conducted for a given time period. In this case the topic has not to be analyzed but is given by the learning context.

Hypothesis2: There is a second summary analysis possible based on the same parameters: Confining the number of attachments and the number of referenced URLs in separated graphs and follow it over time it might be possible to deduce a learning phase as exploration (many URLs) and documentation (attachments).

Number of E-Mails to addressees by one sender Figure 2b presents an example where the occurrences of communicative acts are counted. One E-Mail to a group of receivers counts correspondingly several times. The aspect diagram relates the sender to the receiver. In the graph visualization the peak of an arrow points to the receiver and the end of the edge points to the sender. Again, darker gray tones represent a higher number of communicative acts. In this case the bidirectional relations are only possible if the BCC feature of E-Mails is used, otherwise, merely incoming relations would exist.

Hypothesis1: The occurrences of relations between senders and addressees tell something about the intensity of cooperation. There have to be assumed different kinds of cooperation. A bidirectional and intensive cooperation might reflect a working or learning group, a sort of collaboration. If this collaboration does not correspond to an official working group it indicates a working or learning network.

Hypothesis2: If the cooperation is not bidirectional the number or intensity of communication reflects the role or character of a person. It could mirror a coordinative function or a leading function for a topic. If the intensity is similar for different topics this visualization rather reflects a persons role or character as communicative or supportive. If it differs from topic to topic it might signalize a leading function for a topic development.

Number of E-Mails to addressees by several senders Figure 2c expands the former example by integrating communicative acts of more than one person into one diagram. This results in more complex structures.

Hypothesis1: Again, bidirectional edges indicates cooperation. In order to assign a two person cooperation to a group there have to be made assumptions about the transitivity of cooperation. It is assumed here that a cooperation between person a and person b and a cooperation between person b and person c does not mean that person a cooperates with person c. This means cooperation is not transitive. In case of a cooperating team there are supposed bidirectional communication among each of the group members.

Hypothesis2: A group structure where not each person communicates to the other denotes a ‘best practice’ network.

Hypothesis3: Whenever an unbalanced communication flow between people is detected this again can denote a specific role for people like experts or heads of groups.


80

In order to visualize communication structures the Touch Graph2 tool is used. Touch Graph is the result of an Open Source project maintained at sourceforge3. Touch Graph supports the visualization of graphs and even big graphs. It offers spring-layout, focus and context, zooming, link functionality for graph nodes and node sensitive pop up information. A graph is defined by a XML-file wherefore Touch Graph predefines the syntax in a DTD. For the CommunicAID:CommAn prototype the Touch Graph graph viewer is deployed to show actual communication structures. Therefore, we developed an integrated analysis tool specialized for communication that includes the graph viewer directly.

Figure 3. The interface of the analysis tool ‘CommunicAID:CommAn’.

Figure 4. A part of a network of communicative acts based on E-Mail logs of two persons.

Figure 3 shows the interface of the CommunicAID:CommAn prototype. The menu allows for checking out E-Mails regarding individual users. Each E-Mail user has to accept this analysis by putting in his or her password.

2 http://www.touchgraph.com/index.html 3 http://sourceforge.net/projects/touchgraph/


81

Currently no analysis is performed in the background without explicitly informing the users. The sets of E-Mails can be integrated afterwards or shown one by one. Message filters define the time interval relevant for a single analysis. Together with the edge color caption the analysis type defines how the edges of the visualized graph have to be interpreted. In Figure 3 the quantity filter has been selected. The graph shows the example of Figure 2b)4. Figure 4 shows communicative acts of two persons that already includes some networked connections.

Discussion and Outlook Within this paper the term ‘communication structure’ has been introduced as meaningful situations at a point of time or as activity patterns. Both are visualized using a particular kind of diagram that makes explicit the actual selection of aspects. This does not lead to a meta-model of communication but assumes that an unpredictable set of aspects can influence communication. This paper reflects the starting point of a communication analysis approach. Since only quantitative aspects have yet been considered, it is planned to combine these quantitative analysis aspects with qualitative aspects, such as the aggregation of structures according topics and linguistic criteria e.g. the meaning of a contribution for the communication process.

E-Mail communication is used because it is today probably the most common communication media. Applying the theory of Nonaka (1994), where communication is explained as the main means of knowledge construction, a circular process where implicit information is made explicit, elaborated and internalized, this lead to our working hypothesis that E-Mail communication might reflect knowledge processes as well as group and cooperation structures, topic development and roles of people. Tyler et al. (2003) and Gloor et al. (2003) have successful shown ‘leadership’ as a graph pattern. They used a graph algorithm in order to separate sub graphs that represent communities. The central idea of this work is to add the direction of communicative acts and the frequency in order to identify more patterns according roles, personality traits and cooperation.

Reporting on work-in-progress this paper can not anticipate the criteria that actually support the interpretation and analysis of group cooperation. The analysis approach will be applied to learning and working groups in order to test the hypotheses about cooperation. The tool will be expanded in order to find relevant intervals that show specific peaks of a cooperation as maximal workload or passive learning phases.

Bibliography Collins, A. & Ferguson, W. (1993). Epistemic forms and epistemic games: structures and strategies to guide

inquiry. Educational Psychology, 28(1), 25-42. Gaßner, K. (2003). Diskussionen als Szenario zur Ko-Konstruktion von Wissen mit visuellen Sprachen (Using

the discussion scenario for the co-construction of knowledge with visual languages). Dissertation (PhD), Universität Duisburg-Essen. http://www.ub.uni-duisburg.de/ETD-db/theses/available/duett-03022004-171520/

Gaßner, K., Jansen, M., Harrer, A., Herrmann, K. & Hoppe, H.U. (2003). Analysis methods for collaborative models and activities. In Wasson, B., Ludvigsen, S. & Hoppe, U. (eds.) Proceedings of the International Conference on Computer Supported Collaborative Learning (pp. 369-377). Dordrecht; Boston; London: Kluwer Academic Publishers.

Gloor, P.A., Laubacher, R., Dynes, S.B.C. & Zhao, Y. (2003). Visualization of communication patterns in collaborative innovation networks. Proceedings of the 12th int. Conference on Information and Knowledge Management (pp. 56-60). New Orleans, USA. Lewandowski, T. (1994). Linguistisches Wörterbuch 2 (6.th edition). Heidelberg; Wiesbaden: Quelle & Meyer. McGrath, J.E. (1991). Time, interaction, and performance (TIP): A theory of groups. Small Group Research, 22

(2), 147-174. Nonaka, I. (1994). A Dynamic Theory of Organisational Knowledge Creation. Organization Science, 5 (1), 14-

37. Tyler, J.R., Wilkinson, D. M. & Huberman B.A. (2003). Email as Spectroscopy: Automated discovery of

community structure within organizations. A study by Hewlett Packard: www.hpl.hp.com/research/idl/papers/email/email.pdf

Wellman, B. (2001). Computer networks as social networks. Science, 293, 2031-2034. Wenger, E. (1998). Communities of practice, learning, meaning, and identity. New York: Cambridge University

Press.

4 Colors are reworked to gray tones.


82

http://www.hpl.hp.com/research/idl/papers/email/email.pdf

Improving learning and soft skills using Project Oriented Learning in software engineering courses

Julieta Noguez, Enrique Espinosa Instituto Tecnológico de Estudios Superiores de Monterrey

Campus Ciudad de México. Calle del Puente 222, Col Ex Ejidos de Huipulco, México D.F. CP14380 México D.F.

{jnoguez, enrique.espinosa}@itesm.mx Phone: (52) (55) 54-83-21-80

Abstract.

We present a study that identifies student behavior, development of soft skills and learning improvement during a project oriented software engineering course at the B. Sc. Level. Assessment of behaviour characterized as soft skills and knowledge when students work in a collaborative way is hard to achieve, but useful for effective tutoring. We contribute to constructing a strategy for applying self-assessment on collaborative actions that take place in the classroom, assuming that such actions are the manifestation of the learning process. Project Oriented Learning (POL) considers that students will work on a single guiding thread, or project, for an entire course. We present a research trend that allows the process to be managed, as well as three years of in-class results.

Key words: Project Oriented Learning, Cooperative Learning, Soft skills.

1. Introduction.

There is a worldwide discussion on how to teach engineering students at higher education levels because of society changes and new requirements concerning skills, abilities and ethic values of future engineers. An ever increasing number of academic institutions are undergoing a search new methodologies and didactic techniques which enable undergraduate students to face real professional situations [7].

Students must be scaffold in order to have them perform properly when they must organize themselves as teams, and play roles while delegating work onto themselves, and when delivering feedback to their teams. Overall success in these terms is not easily measurable, since most of the learning process will take place outside the realm of the computer system and will thus have to be assumed whenever there is evidence of its existence through visible actions [4]. Besides it is hard to prove that students are motivated to learn when the instructor applies POL to their classroom activities. Johnson states “… changing to a cooperative style is not simple. There is a big difference between putting students into groups to learn… and structuring your teaching so students learn cooperatively...” [6].

The project oriented technique provides the following advantages [1,10]:

• It allows the students to learn how to solve problems using relevant knowledge independently of the discipline source.

• Activities are focused in exploring and working a practical problem with an unknown solution.


83

• They are designed in such a way that at least last for a curse, they can involve several contents of the same discipline or the interaction of different disciplines.

• They consider in their design the application of different interdisciplinary knowledge so the student can appreciate the relationship between different disciplines in the development of a particular project.

• Allowing the search of open solutions so students are free to create new knowledge.

The Project Oriented Learning (POL) didactic strategy portrays active learning as an educational paradigm that transforms direct experience into a tool for supporting, and stimulating, learning [9,11]. The following descriptions and results, applying POL, are made in the “Software Engineering I” course at the Mexico City Campus, Tecnológico de Monterrey System. The subjects are held for 4th semester students of CS majors. A more detailed description of the POL technique application will be described, as well as the importance of a portfolio in the process of critical thinking of acquired knowledge and abilities [8]. 2. Course Description

The following [general] objectives of the software engineering course were defined:

� Know, understand, and apply the analysis and design methodologies during the development of computerized information systems in organizations.

� Identify problems in the use of information technologies in order to plan, analyze, design, and construct information systems with a creative solution.

To achieve these goals, the student needs lead, and coordinate, so she/he will be able to develop robust and easy

maintainable information systems. Preparing technical documentation and manuals are necessary for the maintenance of information systems. A reflection process is a very important tool for students. They need to construct a portfolio for learning-by-doing management, conflict resolution, and overall synthesis of all products derived from the final team integration and maturity. It also serves to point out elements that have not been completed, and thus contribute to the overcome flaws which may appear throughout the course. 2.1 Soft and technical skills.

Besides of these activities, we chose different soft and technical skills to develop during this course, as shown in table 1.

Table 1. The desired technical and soft skills to work in this course

Soft Skills Technical Skills � Team work. � Leadership. � Responsibility. � Self directed learning. � Honesty. � Management. � Planning. � Negotiation.

� Capacity to identify and solve information management problems in companies.

� Applying tools to model information systems. � Development of an information system (planning,

analysis, design, development, testing, and so on). � Development of technical documentation and

manuals of the system.

2.2 Learning and soft skills approaches.

We have two approaches. We describe the first approach in this course for sophomore developers. The

students need to integrate previous knowledge of basic programming and data structures and to apply software


84

engineering concepts of this course to develop a transaction process system. Because the students have little or no project management experience, teacher planning is needed to have pre-defined milestones and breadth is fixed before hand. The course is in a Learning Content Management System (LCMS) [2]. All the teaching material and support structure (i.e. presented as content in an author static tool called Blackboard) are to be used by the student, while teamwork and assessment, actions that allow learning to actually occur, will be stored as behavior evidence of collaborative actions. In practical terms, a Blackboard course delivery of POL courseware must include facilities for observing and recording concrete experience, observations & reflections, formulation of abstract concepts & generalization, and testing implications of concepts on situations. The students can choose their own workload and the team is responsible of their balancing. The fixed goals related with learning and soft skills are assessed by self-perception.

Our second approach is applied in a course for senior developers. It means the students have project

management experience. We use web software based in a contract, that allows the process to be managed and assessed. The course uses a Computer Supporting Collaborative Work (CSCW) [2]. The authors proposed that a workflow with written compromises, called a Work Contract, is a socially accepted tool because it is modelled as a consulting device. The students should to have a planning project filling in a contract due dates, product list, task competitions, etc. and, the team is responsible of the workload balancing. The web system follows behaviour students by mechanistic perception of the student’s soft skills based on probabilistic reasoning. To see more detail in Espinosa, 2004 [5].

3. Using POL to evidence learning.

During a consulting & learning-type semester, a guiding thread that asks the students to build an information system of medium complexity, and a portfolio of project are running. The portfolio records are made up of the following phases [4]: hiring and contractual agreements, system analysis and design components, integration and recovery units and recapitulation and delivery actions.

The group process to develop the consulting project will consist on establishing the following steps:

1. At the beginning of the course we ask the students to reflect on their expectations towards the course, their actual knowledge level concerning the course and their commitment to contribute to the course success.

2. During the following two weeks the students obtain knowledge about the group building process, responsibilities of a leader, conflict resolution strategies and project management tools

3. The team itself, that is, who conforms it, and what are each member's strengths and weaknesses. 4. Commit contract. From the third week on teams will be formed and start their team project. At this point,

they made and sign a formal inside contract specifying the roles of each participant. Each team contains five members, and during each phase of the semester the leading role should be changed

5. The teams are asked to do weekly reflections on their work, using a portfolio in Blackboard [3]. Some reflections are individual and some collaborative (reflections about: group conflicts, expectations, goals, progress, answer what works properly? are there troubles? what can we improve?).

Three milestones and a final presentation were defined to show project advances and make team reflection. The instructor uses this information to supervise the progress of each team by reading the reflections and inferring soft skills and learning goals. In case of problems, the teacher can interfere in the process depending on the conditions and circumstances of the team. We use a decision support system during POL management on the learning content management. We combine team reflections of self-perception with teacher assessments by technical goals delivered. We show in table 2, examples of some rules to assess group process, also it is shown an example to assess a technical goal delivered. During the process, information system is programming and testing using module techniques. The four-unit course deals with the ability to integrate these, using the project plan and specifications developed in one unit. As a result, the work contract will be modified once again, specifying all changes, revisions, and additions to the architecture and


85

system model. These two, architecture and model, must be partially executable by now, so a set of demos and test cases must be included in the job plan. Success in building a coherent set of test suites will evidence that integration is becoming a reality, or will prove weaknesses in the development plan, the business specs, or the group's performance as a team.

Table 2. Examples of rules to assess group process and technical goals delivered

Group Failure Technical Goals Delivered groupFailure(Gx) iff Read(PROF, WRF) Λ groupAutoperception(Gx, weak teamwork, WRF ) groupFailure(Gx) iff Read(PROF, WRF) Λ groupAutoperception(Gx, good teamwork, WRF ) Λ ¬ techGoalsDelivered(Gx,WRF)

techGoalsDelivered(Gx,WRF) iff GradingConcept(Gx,index(WRF,1),PASS) Λ GradingConcept(Gx,index(WRF,2),PASS) Λ … GradingConcept(Gx,index(WRF,k),PASS) Λ Count(index(WRF,h) �7)

Where: PROF= Professor, WRF= work auto-assessment, Gx=Team, PASS= approved 4. Results.

The pedagogical method has been applied for six consecutive semesters with success. The POL technique and portfolio use have proven to be an excellent tool for improving learning-by-doing, conflict resolution, and overall synthesis of all products derived from the final team integration and maturity. It also serves to point out elements that have not been completed by students, and thus contribute to decreasing team failures relative in the course. A final questionnaire is applied at the end of each semester about different aspects of project oriented approach in this course. In figure 1 we show the students’ opinion about which kind of activities they thought helped them learn better.

Students' opinion about how learn better

0,00%10,00%20,00%30,00%40,00%50,00%60,00%70,00%80,00%90,00%

2001-01 2001-02 2002-01 2002-02 2003-01 2003-02

Semester

Per

cent Collaborative job

Self Job

Teacher Lectures

Figure 1. Students’ opinion about which kind of activities they thought learn better The questionnaires were applied to 60 students enrolled in two groups of 30 students. Eight teams, of 3 to 4

students, were formed in each group. We were refining POL course across the semesters, but in 2002-01 we tried to combine a Data Base course with this Software Engineering course. In this semester the main problem was the difficult to coordinate teachers of different courses (applying POL between teachers was hard).

Also, we asked students about their perception of the soft and technical skills acquired during the course. In figure 2 we show main skills pointed out by students.


86

Students' perception about obtained soft skills during the course

0,00%

20,00%

40,00%

60,00%

80,00%

100,00%

2001-01 2001-02 2002-01 2002-02 2003-01 2003-02

Semester

Per

cent

Capacity to Identify andSolve ProblemsTeam Work

Leadership

Responsibility

Self Directed Learning

Figure 2. Students’ perception about obtained soft skills during the course

A high percentage of students consider that they developed specific soft and technical skills in this course. This question was included in the package of questionnaires described above. Each skill has some variations because we were refining POL course across the semesters. Finally, we were recording the final assessment of knowledge tests and projects like evidence of improving learning. The results are shown in figure 3.

Average results of final assessments

0,00

20,00

40,00

60,00

80,00

100,00

2001-01 2001-02 2002-01 2002-02 2003-01 2003-02

Semester

Gra

des

Final exam

Final Project

Final exam/without POL

Final Project/Without POL

Figure 3. Average results of final assessments

These results were obtained of the same sample of 60 students enrolled in two groups of 30 students. Eight teams, of 3 to 4 students, were formed in each group. We also included information about a 30 students group without apply POL technique. 5. Conclusions and future work.

We presented project oriented learning in a software engineering courses and using it to give evidence

about learning improvement and development of soft skills. The results shown the main advantages of this approach: (i) it improves student learning, (ii) the students become concern about their strong and weak points, (iii) the project quality is improved, and (iv) the students obtain professional and personal skills in strong relation to their future work field

We found some difficulties: (i) the roles of the teacher as consultant, coach and professor still need to be

refined, because is the same teacher attending each course, and the professor still needs to spend a considerable amount of time with the students, both individually, and as teams, or even as groups of teams, (ii) there is a mayor


87

work load for students, (iii), at the beginning of this approach, there was a resistance to change roles (students and teachers), (iv) combining more than one course in a same project is hard, and (v) applying POL between teachers is also hard.

Until now we have two approaches from fixed goals to free goals as you can see in figure 4. We described

in this work, only the first approach.

Figure 4. Learning and soft skills approaches

We would like to look for a convergence of these two approaches in order to improve learning and soft-

skills assessments based in student’s behavior using Project Oriented Learning. 6. References [1] Algreen H. and Moesby E. Assessment Guide for Students. Aalborg University, Denmark. 2001 http://aua2.aua.auc.dk/fakktekn/aalborg/engelsk/index [2] Arevolo W., Lundy J., Phifer G. and etal. Hype Cycle for Corporate e-learning. In Gartner Research. Strategic

Analysis Report. June 25th , 2004. [3] Blackboard is a trademark by Blackboard Inc. http://www.blackboard.com [4] Espinosa E. Noguez J. Assisting Students with POL using XML- Aglet Federation. 47th World Assembly.

Teacher Education and the Achievement Agenda. Amsterdam. Julio, 2002. [5] Espinosa E., Noguez J. Project Oriented Tutoring on Milestone Behavior using contract management. Publishing

in advance in: 34th ASEE/IEEE Frontiers in Education Conference. 2004. [6] Johnson R. How can we put cooperative learning into practice?. The science teacher. Vol 67. No. 1. January

2000. pp 39. [7] Martín M. El modelo educativo del Tecnológico de Monterrey. Ed. Tec de Monterrey. Monterrey, Nuevo León.

México. 2002. pp 17-29. [8] Noguez J. Espinosa E. Using a Portfolio for the Didactical Technique Project Oriented Learning in some

Computer Systems Subjects at ITESM-CCM. 47th World Assembly. Teacher Education and the Achievement Agenda. Amsterdam. Julio, 2002.

[9] Oosterhuis-Geers J.A. BITskills. Education Center. Internal Report. Business Information Technology. University of Twente. Netherland. May 1997.

http://www.unimaas.nl/pbl/general/general001.htm [10] Powell P. and Weenk W. Characteristics of Project Work. Dinkel Institute, University of Twente, Netherland.

2000 [11] Sabine, Dierick. ”Assesment in a problem-based learning environment”. ITESM-PBL Advanced Training

Course. Univ. of Maastricht. June 2000. Lecture notes.

Learning and soft skills approaches

FIXED GOALS

FREE GOALSassessments monitoring analysisBEHAVIOR

LCMSCSCW4th semester

8th semester

Self-perception

Mechanistic perceptionMechanistic perception


88

Mining Techniques for Models of Collaborative Learning

Thereza P. P. Padilha1,2, Leandro M. Almeida1 and João B. M. Alves2 1Lutheran University of Brazil - ULBRA Email: {thereza, leandro}@ulbra-to.br

2Federal University of Santa Catarina - UFSC

Email: {thereza, jbosco}@inf.ufsc.br

Abstract: Many student interactions in the collaborative learning process can be captured and stored in a database for future analysis. However, the precious information extraction in database is almost impossible without the use of mining techniques. In this paper we present a model of collaborative learning, individual and group, using data and text mining techniques. Our model allows us to extract relevant information about collaborative learning interactions at different levels of abstraction.

Keywords: collaborative learning, data mining, interaction data, text mining.

1 Introduction The analysis of collaborative learning interactions is considered a key issue and powerful because it allows us to known and “understand” how learning evolution among students happens, for example. Several computational models of collaborative learning are found in literature, such as finite state machines (McManus & Aiken, 1995) and rule learning (Katz et al., 1999). Each one of these models has a different perspective. A review of some existing models can be found in (Soller & Lesgold, 2000). Before building models, it is necessary to identify the variables that are to be modeled. This is a difficult step because the specific variables that play an important role in this complex process are deeply entangled and, therefore hard to isolate in research (Pol, 2002). The next step towards the building of computational models is to analyze variables values. The analysis process is essential because in interactions’ data (logfile) can be stored unnecessary, missing and redundant data. Thus, the use of mining techniques for processing large amount of logfile is indispensable (Martinez et al., 2002). In this context, this paper presents a model of collaborative learning using data and text mining techniques. The model provides set of performance reports that allow us, for example, to compare a specific group with other groups or a member with the other group’ members and verify the student actions historical. For this purpose, we use data mining (DM) techniques to compare the current state of interaction with the ideal state, and text mining (TM) techniques to identify and categorize contribution types in the dialogs. Our interest is to discuss related issues to models of collaborative learning and, mainly, the question “What compilation or abstraction methods are needed to construct a computational model from a logfile describing the group interaction?”, described in the call for participation.

This paper is organized as follows. In section 2 we describe an overview data and text mining techniques. In section 3 we describe the functionality of the proposed model. In section 4 we present the conclusions and then comment some topics for discussion in the workshop. 2 Data and Text Mining - overview DM is a technique that consists of applying data analysis and discovery algorithms that, under acceptable computational efficiency limitations produce a particular enumeration of patterns (or models) over the data (Fayyad


89

et al., 1996). Data mining has been directed to search patterns from data set using methods such as neural networks, symbolic machine learning algorithms, probabilistic reasoning, etc. In the symbolic algorithms field, actually, there has been much interest in the semi-supervised learning, an intermediate type between supervised and unsupervised learning. In this context, learning refers to rules set, for instance. The semi-supervised learning has as main characteristic the incorporation of background knowledge through labeled examples in unlabeled data set for future learning (Bruce, 2001). Initially, a supervised learner is build using the labeled examples and then applies the trained learner on unlabeled data. There is not a pre-defined amount of labeled examples that should be inserted in database, however, if one database contains a high number of labeled examples more easy and correct will be its works. The semi-supervised learning was chosen because of its flexibility and accuracy to use incorporated knowledge (ideal state), represented by labeled examples in the data set, and to classify the students’ performance, represented by unlabeled examples, in collaborative process. For each realized classification, it is possible to know its accuracy level and the used patterns for definition of the value. Another reason is the ability to work with an undetermined amount of examples, but it is important to provide a minimum quantity of data. TM is a technique that looking for regularities, patterns or trends in natural language text from unstructured or semi-structured texts (Tan, 1999). TM includes several text processing and classification tasks such as text categorization, clustering, summarization, information retrieval, etc. The text categorization, for example, is one task for labeling natural language texts with thematic categories from a predefined set. The categorization is realized via similarity measure assigning a Boolean value to each pair (dj, ci) ∈ D x C, where D is a domain of documents and C = {c1, c2 … , cj} is a set of predefined categories (Sebastiani, 2002). The categories are just symbolic labels. Categorization using Boolean model is simple because only verify, in D, if there is the presence of one or more words stored in C, for instance, to classify it. In the model, we use the text categorization task to identify the student intentions such as task division, decision making and explanation among messages sent. For this identification, we need to build a set of predefined categories to evaluate the semantic. The principal advantage is to eliminate the dependence on users to provide their contribution types. However, for each application domain will require a specific set of category. 3 Mining Techniques to Model Students’ Interactions The proposed model is incorporated in a collaborative problem solving environment, implemented in Java, in which the collaboration is based on five steps that are: reality observation, key-points, theorization, hypothesis elaboration and reality fitting (Padilha, 2003). To facilitate the building model, two computational agents named awareness and collaborative were defined and will be described in more detail. 3.1 Awareness Agent The awareness agent’s goal is to capture, categorize and store several contribution types. The students’ interactions (actions) for problem solving are stored in a MySQL database. We identified a set of quantitative and qualitative variables for modeling. The quantitative variables, basically, inform individual and group interactions number using communication tools or other resources available. On the other hand, the qualitative variables provide a social and cognitive aspect of interaction through actions performed by students. Table 1 presents a brief description of the variables used for modeling. The identification and categorization of the variables were realized with help out from several educators and psychologist. Although awareness agent is implemented, we build a simple data set containing 40 examples about student interactions in the problem solving steps. Afterward, we added 10 labeled examples representing the ideal state. Thus, the data set consisted of 50 examples, in which 80% of unlabeled examples and 20% of labeled examples, and


90

11 attributes being 10 related with quantitative and qualitative variables and 1 to performance value. The quantitative variables values are numeric. The qualitative variables values are categorical that are: low, middle and high. These values are defined by observing and analyzing the group’s conversation and actions in order to identify situations in which students effectively acquired knowledge. For this analysis, we determine a set of 5 predefined categories (organization, argumentation, information, request and motivation) to use the text categorization task. Each one of these categories is associated with one qualitative variable. The organization category supports the task division variable; argumentation and request support explanation variable; information supports decision making variable; and motivation supports involvement variable. Every category consists of, in average, 20 words.

To improve performance in the categorization and reduce search time, a selection process is realized for find adequate words. So, when a message is analyzed, “irrelevant” words (stopwords), such as articles and prepositions, are ignored. For example, the message “How will we do the work?” has a high probability for task division variable because it has the “work” private word belongs to organization category and the adverb “how”. The implementation of the text categorization follows methods presented in (Vapnik, 1998). 3.2 Collaboration Agent The collaboration agent works to build the model and produce some performance reports. First, the labeled examples are provided to a learning algorithm that generates a learner (patterns) analyzing its attributes’ values. The patterns are represented as production rules, i.e. if-then format. Second, the unlabeled examples are submitted to rules that classify its performance. The performance value is numeric between 0 (bad) and 10 (excellent). The pattern discovery process is based in the Entropy and Gain Ratio methods. The Entropy is responsible to measure disorder level in attributes (Williamson, 2002). Entropy is high if there is a lot of disorder, otherwise it is low. Gain Ratio is responsible to obtain the amount of relevant information in a specific attribute (Quinlan, 1996).

Table 1: Variables for Modeling Type Name Goal Actions

Chat

To obtain the interactions number in synchronous discussion sessions in chat tool.

The computation of this variable occurs when the students send messages, use sentences openers, and access last discussions. Messages without content, with repetition of only one character or high number of symbols are not computed.

Text Editor To obtain the interactions number in the textual edition tool.

The computation of this variable occurs when the students create blocks in exist files, create a new file, read, write or change texts.

Vote To obtain the interactions number in vote tool, directed, mainly, to decide ways that the students should follow.

The computation of this variable occurs when the students create a new voting or vote in one current voting.

Form

To obtain the number of participations in the form filling, needed for problem solving.

The computation of this variable occurs when the students forms filling for problem solving.

Q u a n t i t i v e

Repository To obtain the interactions number in the repository.

The computation of this variable occurs when the students make document upload or download.


91

Decision Making

To identify the degree of decision making for problem solving.

The computation of this variable occurs through a semantic analysis in messages sent.

Task Division

To identify the degree of organization among students to divide theirs tasks.


Regularity

To identify the degree of access to environment and tools during problem solving as a whole. The regularity reflects the interest and responsibility with other group members.

The computation of this variable occurs when the students access and use environment, communication tools, and other resources available.

Explanation

To identify the degree of explanation/argumentation by students for discussing related topics with problem.


Q u a l i t a t i v e

Involvement

To identify the degree of interactivity among group members. The involvement is a variable very important to indicate the presence of communication among students.

The computation of this variable occurs through an analysis of interactions realized among students using communication tools and other available resources. However, if a student has high value in use of the chat tool but not in vote tool, then he/she will be considered medium involvement. Moreover, a semantic analysis occurs in messages sent.

The collaboration agent provides a set of reports presenting an overview of the student performance. The reports have different levels of abstraction: comparing a specific group with other groups, comparing a member with other group’ members and analyzing of student actions historical. The Figure 1(A) presents, graphically, the group performance report during the solving of four problems. For each existing group, there is a specific line format that it is possible to observe its performance and verify the global performance (available in the below legend). The group B, for example, had the bad performance as can be seen in its line format. The student performance report is very similar to group performance report. In this case, it is possible to verify the performance of each student in a group. The Figure 1(B) shows the student performance report of the group A.

Figure 1: Examples of Performance Reports

(A)

(B) (C)


92

The Figure 1(C) illustrates, in more details, all students’ actions in key-points step of the problem solving (Problem #1), describing what, who and when a determined action was executed. In addition, it is possible to see the quantitative and qualitative variables values for each group’s member. The performance reports offer a multiple perspective view of the collaborative learning process. Group performance reports simplify the nature of interaction among existing groups. Student performance reports help to explain because a specific group had a below performance (value 2), for example. The report’s representation form, graphic, facilitates also its general analysis. Moreover, there are recourses available to query quantitative and qualitative variables values for each student. 4 Conclusion The manner to capture and store student actions in joint process is essential for designing of consistent collaborative models. The paper presented how mining techniques can help in the processing student interactions data and determining of the learning performance. The text mining technique can be seen as an alternative for understanding of conversations patterns among students, without that they need to define their contribution types via sentence openers avoiding a possible error. The categorization of contribution types by awareness agent has demonstrated a reasonable performance because the natural language has many ambiguity and still need of human interpretation. In our model, it is necessary to build a complete set of categories. For each existing category, many words and terms should be associated. The data mining technique used, semi-supervised learning, is very useful to compare the current state of interaction to ideal state because of the possibility for background knowledge incorporation. According to realized experiments, the model has shown some relevant results for analyzing student performance. The results are not totally accurate because we do not have still a real data set that expresses information related to problem solving. Some refinements are being realized to improve the semantic analysis in messages and offer mechanisms for supporting inferences. Topics for Discussion in the Workshop Our group is working in the designing of computational models of collaborative learning interaction using mining techniques. With this perspective, we have some questions for discussion in the workshop:

• Are there any advantages for the definition of standard representation of collaborative models? Robustness? Extensibility? Information interchange? Interactivity? Is there already an initiative for standardization of the model?

• Recently, XML bas been proposed as the standard data representation for many applications. What are some advantages and disadvantages of using XML as a collaborative learning representation language? Only for possible access in the elements of the model?

• Which steps a system should be able to perform with interactions data before building models? Data cleaning? Selection of relevant examples? Treatment with missing values? Any methodology?

• From individual interaction data set is it possible to predict the future group performance? To possibility new inferences? Uncertainty?

• Why is it important that the user does not intervene in the categorization of the dialogue? • What are the problems that have to be addressed when using TM technique for categorizing texts in the

context of CSCL applications? References Bruce, R. (2001). Semi-supervised Learning Using Prior Probabilities and EM. Proceedings of the IJCAI Workshop on Text Learning. pp. 17-22. Fayyad, U., Shapiro, G. P. and Smyth, P. (1996). From Data Mining to Knowledge Discovery in Databases. AAAIMIT Press, pp.37-54.


93

Katz, S., Aronis, J. and Creitz, C. (1999). Modelling Pedagogical Interactions with Machine Learning. In S.P. LaJoie & M. Vivet (Eds.), Proceedings of the Ninth International Conference on Artificial Intelligence in Education, LeMans, France, pp. 543-550. Jermann, P., Soller, A., and Muehlenbrock, M. (2001). From Mirroring to Guiding: a Review of State of the Art Technology for Supporting Collaborative Learning. pp. 324-331 Martinez, A., Marcos, J.A., Garrachon, I., Fuente, P. and Dimitriadis, Y. (2002) Towards a Data Model for the Evaluation of Participatory Aspects of Collaborative Learning. CSCL 2002 Workshop: Designing Computational Models of Collaborative Learning Interaction, Boulder, Colorado. McManus, M. and Aiken, R. (1995). Monitoring Computer-based Collaborative Problem Solving. Journal of Artificial Intelligence in Education 6, 4, pp. 307-336. Quinlan J. R. (1996). Improved Use of Continuous Attributes in C4.5. Journal of Artificial Intelligence Research: pp.77-90. Padilha, T. P. P. (2003). Um Ambiente de Aprendizado Colaborativo para Resolução de Problemas, Monografia de Qualificação de Doutorado, CPGCC, UFSC. Pol, J. (2002). Identifying and Modeling Variables in Complex CSCL-situations - Case Study: The Use of Asynchronous Electronic Discussions. CSCL 2002 Workshop: Designing Computational Models of Collaborative Learning Interaction, Boulder, Colorado. Sebastiani, F. (2002). Machine Learning in Automated Text Categorization. ACM Computing Surveys, Vol. 34, No. 1, pp. 1-47. Soller, A. and Lesgold, A. (2000). Modeling the Process of Collaborative Learning. International Workshop on New Technologies in Collaborative Learning, Awaji-Yumebutai, Japan. Tan, A. H. (1999). Text Mining: The State of The Art and The Challenges. In Proceedings of the PAKDD’99 workshop on Knowledge Discovery from Advanced Databases, Beijing, pp. 65-70. Vapnik, V. (1998). Statistical Learning Theory. New York: John Wiley&Sons. Wagstaff, K., Cardie, C., Rogers, S. and Schroedl, S. (2001). Constrained K-means Clustering with Background Knowledge. In Proceedings of the Eighteenth International Conference on Machine Learning – ICML, Williamstown, pp. 577-584. Williamson, J. (2002). Maximizing Entropy Efficiently. In the 19th Workshop on Machine Intelligence, Imperial College at Wye, Department of Philosophy, King's College London.


94

ANIMATED-CHAT Facial expression to support sense of presence

Luciane Maria Fadel Adalberto Nazareth

[email protected]

[email protected]

Abstract: In this paper, we describe a graphic interface for chat rooms in an online education environment that

uses animated faces to express emotion and thereby convey a sense of presence. The results of a first experiment using different layouts indicated that the use of animated faces helped participants to express themselves. Some of the findings of this experiment were used to propose a new layout, which will be tested.

Key words: Sense of presence, facial expression, chat, online education

1. INTRODUCTION

Human beings are essentially sociable. An individual grows up amongst other people, first as a member of a family and then as a member of a society. An individual learns by observing and interacting with others (Argyle, 1976). As Gunawardena stated ‘we tend to under emphasise the fact that two kinds of knowledge creation take place in any shared learning experience, the ‘ individual’ and the social’. Knowledge is crested at the social –…- and the individual also creates his own understanding by interacting with the group shared construction’ (Niven et al., 2002). But for an individual to share is necessary to interact. It is through interaction that a sense of presence, of being part of a group, is constructed. With a sense of presence, a community can be established (Preece, 2000).

When education is delivered by computing network, students’ interactions will be based on text and sense of presence will be hard to achieve. The lack of visual signals, especially body expressions, may disrupt communication. This disruption can negatively influence interactions, and social presence will be blurred. Therefore, enhancing non-based text interactions becomes crucial to build a sense of presence that leads to a community, which in turn promotes learning. This paper describes a graphic interface for chat rooms where students are able to emote expression through animated faces.


95


2. SOCIAL PRESENCE

There are a wide variety of computer tools dedicated to Online Education (OE). They are processes of Computer Mediated Communication (CMC) used to exchange information, where interaction is a distinguishing characteristic (Miltiadou & Savenye, 2003). Therefore, they support communication between groups or peers. The most popular forms are based on text. Email, forums, newsgroups, instant messaging and chat rooms provide asynchronous and synchronous communication on OE environments. Synchronous communications like instant messaging and chat rooms are usually used for debates moderated by a tutor and social meetings not task-oriented. Chat rooms that permit more than two people at the same time in a virtual room are places to interact socially and make friends. While chat rooms are designed to be places where people get together, their communication is shaped by the restrictions of the medium. Not having face-to-face contact means that participants meet each other through digital media, with no visual cues or sound cues, and no sense of smell or touch. Their interactions are textual. As Palloff and Pratt stated ‘it’s always important to remember that in the online environment we present ourselves in text. Because it is a flat medium, we need to make an extra effort to humanize the environment’ (Niven et al., 2002). The sense of presence in a chat group is restricted by a name and is dependent on active participation, i.e., posting messages. As stated by Ubon and Kimble ‘social presence is one of the most important factors that helps people actively collaborate thus increasing sense of belonging and social cohesion to the community’ (Ubon & Kimble, 2003). Social presence is the sense of being a member of a group with the possibility of interaction. Therefore, social presence is a main factor that leads to active participation in a chat room. When a member of a group becomes more interactive his social presence is better defined and he becomes an insider member or a full participant, of a community (Kreijns et al., 2002). Quoting Jo Kim ‘at the most basic level, a community is a place where everybody knows your name, learns things about you – your personal history, special talents, social reputation, and peculiar quirks – and incorporate that knowledge into their interactions with you’ (Niven et al., 2002). Through interaction students have a sense of presence that in turns leads to a construction of a community. A community of learning, or a community of practice is an ideal place for learning, ‘ …it become crucial to those that recognize knowledge as a key asset’ (Wenger, 1998). Hence, to achieve a high level of satisfaction in Online Education, a community of practice must be constructed. ‘Forming a sense of community, where people feel they will be treated sympathetically by their fellows, seems to be a necessary first step for collaborative learning’ (Wegerif, 1998). To construct a community it is necessary to have sense of presence. Sense of presence can be achieved through expression of emotions, mood and feelings.

3. ANIMATED CHAT

Animated Chat is a graphical interface of synchronous communication that uses animated face icons to indicate and support social presence. The Chat has a restriction of 8 participants at a time.

A face disposed on an elliptical form represents each participant. His/her face icon is placed on the left-centre and is filled with a colour. When a participant posts a message his turn is highlighted by an increase in length the border of his corresponding face, and the text appears inside the ellipse delimited by the faces.


96

On the left side of the interface there is a semi-circle with six emotions represented by facial expressions: surprise, joy, anger, fear, disgust and sadness. On the middle is a neutral facial expression. A bar of 10 points scale enables participants to choose the intensity of an emotion. The intensity appears to others in a 10-points scale that surrounds the face.

To ask to ‘speak’ a participant can ‘raise’ his hand. At this moment a hand icon will appear at the side of his face. It will have a number inside that represents his/her order of cue to talk. This feature depends on a co-ordinator attitude assumed by one of the participants, to address the queue.

Emotions can be used to reinforce a message or to express a feeling about others utterances, and to participate without having to post messages. The participant’s name and face would reinforce his/her sense of presence while all others’ faces would give an exact idea of how many people are participating. The label that holds the name has the same colour as the small circles of the intensity scale.

When there are only two people chatting each face is located by on opposite sides. Each time a new participant enters the room, he/she is located at the left centre position, while to others he/she will appear in a different position depending on the order of entering.

Figure 1. Animated-CHAT room (layout)


97

4. EXPERIMENT

4.1 Overview

This experiment was done using a different interface layout. It enabled only two participants and the faces were based on animated Muppets, in this case a rabbit. Each participant could choose between the six emotions but the intensity scale was not present.

4.2 Participants

The participants were invited to participate. They were high school or undergraduate students at Universidade do Oeste de Santa Catarina (UNOESC). Twenty students took part in the experiment, a pair at a time.

4.3 Procedure

A general explanation about what the experiment was for, and how were the basic functions of the interface were explained to both students before they start. They had 15 min to discuss a topic that was proposed to then. Students were then placed in different laboratories where one observer took notes about their reactions.

4.4 Instrumentation

Participants completed two questionnaires, one at the beginning with demographic questions and one at the end about their perception about the interface. The notes from both observers were compared to the answers of each participant.

4.5 Data analysis

The majority of the participants (95%) liked to use emotions with text entry. This finding was anchored by the observation that the participants used a lot of the animated faces. They usually used it while they were waiting for the answer. They said that the use of the faces helped them to express (80%). Another finding from the observation made was that the participants expressed the same emotion that they chose for the Muppet. When asked if the expression of emotions helped them to express themselves 80% answered positively.

5. CONCLUSION

The results of the experiment were used to design a new layout where more than two participants could get together and flexibility of expression was available. Even though sense of presence was not a topic of research on this experiment because only peers were interacting, it became clear that expression of emotions was used to reinforce the message, to communicate, and so to establish social interaction. This new interface needs to be tested to verify if the sense of presence on group discussion using chat room becomes stronger with the possibility of facial expression, and what are the side effects of it.


98

6. REFERENCES

Argyle, M. (1976) Social interaction (London, Tavistock publications).

Kreijns, K., Kirschner, P.A. & Jochems, W. (2002) The sociability of computer-supported collaborative learning environments, Educational Technology & Society, 5(1).

Miltiadou, M. & Savenye, W.C. (2003) Applying social cognitive constructs of motivation to enhance student success in online distance education, Educational Technology Review, 11(1), pp. 78-95.

Niven, J., Harris, R.A. & Williams, D. (2002) Motivation to use online learning communities: a methodological outlineNetwork Learning Conference

Preece, J. (2000) Online communities design usability, supporting sociability (Chichester, John Wiley & Sons).

Ubon, A.N. & Kimble, C. (2003) Supporting the creation of social presence in online learning communities using asynchronous text-based CMC3rd International conference on technology in teaching and learning in higher education (Heildelberg, Germany,

Wegerif, R. (1998) The social dimension of asynchronous learning network, JALN, 2(1), pp. 34-49.

Wenger, E. (1998) Communities of practice. Learning as a social systems, Systems Thinker.


99

Documents

The 2 Designing Computational Models of Collaborative Learning … · 2017-10-27 · Designing Computational Models of Collaborative Learning Interaction: Introduction to the Workshop