8/3/2019 Evaluating the effect of an ICT educational activity on student success – A three-step approach

http://slidepdf.com/reader/full/evaluating-the-effect-of-an-ict-educational-activity-on-student-success- 1/4

1

ICT and Academic Achievement 

Evaluating the effect of an ICT

educational activity on student successA three-step approach 

Introduction

In his IntersTICES model, Jacques Viens proposes that anyone responsible for theintegration of an ICT educational project must ensure that there is consistencybetween:

  target objectives  actions implemented  resources used  evaluation of results

The three-step process presented here facilitates establishing this consistency:

Step 1: Objectives, activities and resources

  Specify the objectives that will allow the goal (improving academicachievement) to be attained 

  Specify the activities and resources put in place in a coherent manner with a

view to attaining these objectives 

Step 2: Formulating questions and gathering data

  Formulate questions that evaluate conditions and effects   Develop response scales   Gather data from surveys, interviews, on-site observations and productions 

analysis   Compile results and reports in the response scales 

Step 3: Observations, recommendations and dissemination

  Recast response scale results as statistics   Formulate observations and recommendations   Disseminate results and challenges

8/3/2019 Evaluating the effect of an ICT educational activity on student success – A three-step approach

http://slidepdf.com/reader/full/evaluating-the-effect-of-an-ict-educational-activity-on-student-success- 2/4

2

Step 1: Objectives, activities and resources

In order to evaluate the effect of an ICT educational activity on academicachievement, we first need to highlight the initial objectives of this use of ICT.

If we posit that academic achievement is the goal of an experimental ICT project, it

approaches what the Carrefour de la réussite calls a “support measure “: 

A support measure is a set of coordinated approaches that aim for a result or objective related toachievement; moreover, as with all measures, it is an approach specifically designed to “solve” aquestion or problem.

Because of this definition, based on the concept of measures, what distinguishes a "supportmeasure" is its structured nature, organized explicitly, according to results to be achieved in relationto an actual or potential obstacle to academic success. (p.12 - translation)

To assess the effect of the educational use of ICT on student achievement, we mustformulate, structure, and clearly organize the objectives to be met at the time of itsimplementation. Objectives are specific actions put in place to achieve a goal. The

cumulative and combined effects of the objectives allow us to achieve this goal.

ARC’s meta-synthesis of the results of experimental teaching strategies that use ICT toimprove student achievement revealed that research conducted in the college systemhas, to date, focused on objectives related to:

1.  Student grades2.  Academic motivation3.  Complex cognitive processes

These three dimensions can be used as a starting point for identifying the objectives ofan action involving ICT.

The implementation of structured and organized actions involves strategic choiceswithin the organizational constraints. Making the right choices is a major challenge inachieving the objectives. The analysis grid for an ICT learning activity, resulting fromARC’s meta-synthesis and the work of the community of IT Representatives, helps guideand evaluate the choices made in the context of a specific project. Its use, preferablymade at the time of project design, helps ensure consistency between the objectives,activities and resources, as recommended by Jacques Viens. 

The work highlighting the consistency between the original objectives, activities andresources involves the initiators of the original project as well as others who will beresponsible, within a workgroup, to complete all evaluation operations. The expected

product is a synthetic representation in the form of drawings or diagrams, design ofactions (objectives), the conditions of their implementation and the expected effectson student achievement, academic motivation, complex cognitive operations, or other dimensions of student success. An explanatory text or presentation completes thesummary view.

Top of document 

8/3/2019 Evaluating the effect of an ICT educational activity on student success – A three-step approach

http://slidepdf.com/reader/full/evaluating-the-effect-of-an-ict-educational-activity-on-student-success- 3/4

3

Step 2: Formulating questions and gathering data

Once the original objectives and the conditions of their implementation have beenidentified, the workgroup may formulate questions related to their achievement. Clear signs of the expected effects are the indicators of goal achievement. The questionstherefore serve to guide the observation of observable effects. Obviously, the effects ofimplemented actions can occur to varying degrees (more or less often, in a greater or lower number people, to a greater or lesser extent), so that the objectives are in effectmore or less achieved. Designing scales of possible answers to the questions serves totranslate the range of observations.

The purpose of data collection is to answer questions and to "measure" theachievement of initial objectives based on the response scales.

There are several ways to collect data; the conditions of each experiment may dictatewhich method is best.

Among the most common methods, we can consider the following:

  Survey questionnaires for students, teachers or other important players   Group interviews of teachers, students and, eventually, advisors and IT

technicians

  Comments on-site, that is to say most often in the classroom and laboratory,which focus on tasks, the use of space, the definition of roles and interactions 

  Productions by students that serve to analyze their characteristics in terms of theobjectives. 

The report of data collected in the response scales is a crucial step because it oftenrequires that a judgment be made. This task must be performed by more than oneperson, under the principle of triangulation in data coding.

Top of document 

8/3/2019 Evaluating the effect of an ICT educational activity on student success – A three-step approach

http://slidepdf.com/reader/full/evaluating-the-effect-of-an-ict-educational-activity-on-student-success- 4/4

4

Step 3: Observations, recommendations and dissemination

Response scales easily lend themselves to simple statistical processing (mean, standarddeviation, mode). It draws out general information relevant to the questions about theachievement of initial objectives. Has the use of ICT had the desired impact? On alltargets? Which target was best achieved, and which least?

If they lead to clear conclusions, these initial observations can result inrecommendations to improve the next experiments in educational integration of ICT. Ifthe amount of data permits, cause and effect can be explored by statistical tests: isthere a significant difference between the observed effects of this or that group, or onecondition and another?

Some observations may not be conclusive, lending themselves to variousinterpretations. The resulting recommendations may invite further investigation ofpossible links between certain conditions and results.

The compilation of collected data, the observations made of them and the resultingrecommendations are of great value for those involved in the experiment beingevaluated: teachers, techno-pedagogical counsellors, technical teams and servicemanagement. Therefore it is important to share this information through a variety ofmedia, both conventional (reports, articles) and innovative (blogs, videoconferencing). The dissemination of experiment results in educational ICT integrationare of interest not only to primary actors, but also to members of professionalcommunities working in the fields of techno-pedagogy, education, training and,academic achievement.

Top of document