ASSESSMENT OF THE IMPROVEMENT IN RESULTS THROUGH THE ADAPTATION TO THE EUROPEAN

HIGHER EDUCATION SPACE PILOT PLANS

Laura Bravo Sánchez (1), Mª Angeles Cavia Soto (2), Eduardo Mora Monte (1) 1Dep. of Applied Mathematics and Computation Sciences. Cantabria University.

2Dep. of Electric Ingeniering and Energetic. Cantabria University.

laura.bravo@unican.es , maria.cavia@unican.es , eduardo.mora@unican.es

Abstract What is referred to as the European Higher

Education Space is likely to introduce significant

changes in teaching methodology. Its possible

consequences have been analysed through ‘Pilot

Plans’ such as those described here.

In order to appreciate its effect, the present study

focuses on the average success rate indicator over

the number of students enrolled. It can, accordingly,

be stated that this indicator has significantly

improved in the first years of two different

Engineering Degrees. In order to avoid the effect

caused by having subjects of different lengths, the

unit of measure used is the ‘credit’ (10 hours class)

rather than the number of students and the

information has been grouped into two blocks,

before and after the Pilot Plan.

A significant improvement has been shown in both

Degree courses, especially so in three-year Degree,

which can basically be attributed to the smaller

group size and having worse indicator values.

1. Introduction

In the coming years, as a consequence of the agreements signed by the European Union countries on the European Higher Education Space (EHES), there are going to be enormous changes in university studies. In addition to bringing about modifications with respect both to the duration and content of study plans, this process also implies a new focus on teaching methods; these have, up to now, been predominantly centred around lectures in which the students receive a series of subject matters that are to be assimilated, and final exams in which the degree of acquired knowledge is evaluated. The aim of the EHES is that the teaching be specially oriented toward students and their learning, requiring that there be active participation on the part of the students, who so far have done little more than take notes based on what lecturers have said, in order to measure their knowledge at a later specific date.

In the face of these future expectations, the University of Cantabria considered, in the second

semester of the academic year 2004-2005, that it would be convenient to test out the incoming adaptations before the new study plans were actually put into effect. The School of Industrial Engineering and Telecommunications chose to take part in this initiative, starting with the first complete academic year, and so, in the academic year 2005-2006, the Adaptation to the European Higher Education Space Pilot Plans were started in Chemical Engineering and Technical Industrial Engineering, specialising in Electricity. The first is a five-year Degree, and the second is a short cycle three-year Degree.

2. Innovations introduced into the

teaching methods

In most university centres the teaching system corresponded to a model based on lectures being given by the professor, with barely any active participation, if any, on the part of the students, who were later subject to a single exam leading to the final grade. Thus, the ‘credit’ unit was equivalent to 10 hours of the lecturer’s work. A new concept of the European Credit Transfer System (ECTS) referring to the teaching load of subjects is now being postulated, in which the focus is on the number of hours of work the student must take on in order to pass a subject, and includes contact class hours.on in order to pass a subject, and includes contact class hours. In order to make the methodological transfer more simple, most centres have adopted the following equivalence:

‘1 present credit (Spanish Official State Bulletin: BOE) is equivalent to 25 hours of work’ This now means that for each of the subjects taking part in the Pilot Plan learning is more heavily weighted than lectures and this, therefore, calls for a review of how subjects are to be given, an expansion of learning activities and a new means of assessment, where the traditional system of evaluation, by means of a single exam, has less weight than continuous assessment and the control of the student learning process. The following methodology was, therefore, proposed:

Eighth IEEE International Conference on Advanced Learning Technologies

978-0-7695-3167-0/08 $25.00 © 2008 IEEEDOI 10.1109/ICALT.2008.316

225

1. Attendance lectures: contact/ cooperation between the lecturer and students, in two different aspects:

1.1 Lectures (L): the lecturer gives the theoretical content of the subject

1.2 Tutorial Classes (TC): the lecturer supervises and guides the student 2. Non-Attendance Classes: where there is no

direct interaction between the lecturer and the student, with a distinction being made between:

2.1 Tutored Activities (TA): the student carries out tasks planned and directed by the lecturer either individually or in groups

2.2 Independent Activities (IA): the student’s study or personal work time

3. Analysis of the observed results.

Having applied a change in methodology, the first question to consider is whether this change has led to an improvement in academic results. This study has focused on the first years of both degree courses. As the subjects are of different lengths (number of credits) and have different numbers of students, in order to gain overall information for each academic year and for the data to be in accordance with the relevance of each subject, the credits per student for all the subjects have been added together to evaluate the mean success rate over number of students enrolled (quotient between the total number of credits gained by students and the total number of credits for all those enrolled). In order to avoid as far as possible the variability arising between one year and another when it comes to evaluating the corresponding proportion of passes, before the new methods are applied, several previous years have been included in the calculations, these being 5 in the case of Chemical Engineering and 4 for Technical Industrial Engineering, specialising in Electricity. Analogously, the only available information on the application of the new methods, that of the last two academic years, has been used.

So, as our objective is to observe whether there has in fact been an improvement in the results, non

parametric bilateral contrasts of the equality of two proportions (that for the old method and that for the new one) have been performed for each. The contingency tables obtained are those in Tables 1 and 2.

These data lead to the following values of the statistic to be used in the contrast:

Chemical Engineering............................21.10 Technical Industrial Engineering, specialising in Electricity...............................................77.96 From the Chi-square distribution with one degree

of freedom, we get χ2 = 7.88 for a level of

significance of 0.005. As this value is surpassed in both cases, it can be stated that the success rate is different (better) to the previous one with an error probability of less than 0.5%

It is an improvement in both cases since the proportions observed are 0.4574 (from 2000 to 2005) and 0.4874 (from 2005 to 2007) for Chemical Engineering, and 0.4040 (from 2001 to 2005) and 0.4872 (from 2005 to 2007) for Technical Industrial Engineering, specialising in Electricity.

From the values obtained it can also be deduced that the improvement is more marked in the case of Technical Engineering, which can mainly be explained by the average size of the class groups, around 25 to 35 in Technical Engineering and almost double that in Chemical Engineering.

4. Comments on Overall Graphs

Figures 1 and 2 represent the observed proportions of credit passes over students enrolled over a period of several academic years; of these years, the new teaching methodology has been used in the last two, while the traditional methodology of teaching was used in all previous ones. These graphs show that there is a slight variability from one year to another, which is why, in order to provide the proportion contrast, all the information for both periods has been put together when estimating the proportions.

Chemical Engineering

Credit

passes Credit no

passes Total Cred.Pas./Total

Ac.Year

00-05 10276,5 12190,0 22466,5 0,4574

Ac.Year

05-07 3837,0 4035,0 7872,0 0,4874

Total 14113,5 16225,0 30338,5 Statistic 21,10

Table 1: Chemical Engineering Contingency Table

226

Technical Industrial Engineering, (Electricity)

Credit

passes

Credit no

passes Total Cred.Pas./Total

Ac.Year

01-05 3895,50 5748,00 9643,50 0,4040

Ac.Year

05-07 1879,50 1978,50 3858,00 0,4872

Total 5775,00 7726,50 13501,50 Statistic 77,96

Table 2: Technical Industrial Engineering (Electricity) Contingency Table

Figure 2. Evolution of the success rate from 2001 to

2007 in Technical Industrial Engineering,

specialising in Electricity

Figure 1. Evolution of the success rate from 2000 to

2007 in Chemical Engineering

5. Conclusions As a consequence of the European Higher

Education Space, many countries, such as Spain, will undergo important changes in teaching methodology. It is, therefore, clearly of interest to start analysing the possible consequences as soon as one can; this is what the ‘Pilot Plans’, described here, have attempted to do.

In order to establish whether or nor these methodological changes produce an improvement in results, the present study has focused on one of the indicators that best reflect them: the average success rate over students enrolled. The objective here has been to be able to state whether this indicator has actually undergone a significant improvement in the first years

of two different Engineering Degrees, one lasting three years and the other five.

As an academic year consists of several subjects, each with a different number of students and even of varying lengths, then the unit of measure chosen has been the ‘credit’ (10 hours of class) rather than the number of students (passes and enrolled).

In addition, as the time effect produces a slight variability of the indicator from year to year, the information has been grouped into two blocks covering several years, before and after the Pilot Plan.

From the data observed, it can be stated that there has been a significant improvement in both cases, with a negligible error probability (far lower than 0.005). This improvement has been more marked in the case of the three-year Degree, which is mainly due to it having a smaller group size and worse indicator values. Moreover, the fact that the observed rate in the Pilot Plan is almost identical in both cases, which indicates that the degree of difficulty has become equal, could well be the focus of further studies.

6. References [1] “Desarrollo de experiencias conjuntas en la implantación de planes piloto de adaptación al EEES” L. Bravo, Mª A. Cavia, P. Sánchez y Mª F. San Román. IV Jornadas de Redes en Investigación en Docencia Universitaria. Universidad de Alicante (Spain) 5 y 6 de Junio de 2006 [2] “Implementación y desarrollo de un curso completo, en estudios de ingeniería, adaptados con metodología del EEES.” Bravo Sánchez, Laura(1); Cavia Soto, Mª de los Ángeles(2); San Román San Emeterio, Mª Fresnedo(3); Sánchez Barrios, Paulino(4).CUEIEET, Gijón (Spain) 21, 28 y 29 Septiembre 2006 [3] “Terminología relativa al EEES” Mª de los Angeles Martinez Ruiz, Narciso Sauleda Parés Universidad de Alicante y Generalitat Valenciana Ed. Marfil, 2007. [4] Guías Académicas de los Planes Piloto de las Titulaciones de Ingeniero Químico e Ingeniero Técnico Industrial (Electricidad). ETSIIT, Universidad de Cantabria. (Spain).

227