TRAINING ACTION Nº 2 SATISFACTION … General Report.pdf · Higher-Education Partners from ... 561735-EPP-1-2015-1 ... Figure 1 sums up the results of participants’ answers to

____________________________________________________________________________________

[VISIR+] 561735-EPP-1-2015-1-PT-EPPKA2-CBHE-JP

TA2 SATISFACTION QUESTIONNAIRE REPORT 1

TRAINING ACTION Nº 2

SATISFACTION QUESTIONNAIRE REPORT

María Isabel Pozzo & Elsa Dobboletta

(IRICE-CONICET, Argentina)

October, 2016

Introduction

The present Report presents the analysis of the Satisfaction Questionnaires applied

during the Training Actions N° 2 of VISIR+ Project “Educational Modules for Electric

and Electronic Circuits Theory and Practice following an Enquiry-based Teaching and

Learning Methodology supported by VISIR” which took place in the Latin American

Higher-Education Partners from August 22nd to September 16th, 2016.

The Satisfaction Questionnaire was devised as a data collection instrument to evaluate

the impact of the three training actions (TA1, TA2 and TA3) which take place along the

development of VISIR+ Project. The tool was already implemented during the TA1 in

Karlskrona (Sweden, February 2016) and the results were subsequently analysed and

reported. In the case of the Training Actions 2, the questionnaire was given to all

participants at the end of each HE meeting, different from the single TA1 where

attendees answered the questionnaire at the end of each day module. TA2 did not

have virtual participants.

The TA2 took place in five VISIR+ Latin American higher-education institutions, three

host Partner universities from Brazil:

1. Federal Institute of Education, Science and Technology of Santa Catarina

(IFSC),

2. Universidad Federal de Santa Catarina (UFSC),

3. Pontifícia Universidade Católica do Rio (PUCRio),

and two host Partner universities from Argentina.

1. Universidad Nacional de Santiago del Estero (UNSE),

2. Universidad Nacional de Rosario (UNR).

____________________________________________________________________________________



The main lecturers for each TA2 were from European Partner universities:

1. Instituto Politécnico do Porto and Instituto Superior de Enghenharia do Porto

(IPP-ISEP), Portugal

2. Universidad Nacional de Educación a Distancia (UNED), Spain

3. Universidad de la Iglesia de Deusto (UDeusto), Spain

4. Carinthia University of Applied Sciences (CUAS), Austria.

The distribution of lecturers for each LA HE institution was:

Host Institutions Guest Institutions City

UFSC IPP/ISEP Araranguá

IFSC IPP/ISEP Florianópolis

PUCRio CUAS Rio do Janeiro

UNSE UNED Santiago del Estero

UNR UDeusto Rosario

About the instrument

The instrument was designed by the members of VISIR+ Project Workpackage 3 who

are researchers from Research Institute of Education Sciences (IRICE) from the

National Technical and Scientific Research Council (CONICET) from Argentina and

from the Instituto Politécnico do Porto (IPP) from Portugal.

The Satisfaction Questionnaire has 8 closed questions and 1 open question, all

questions expressed in statements about the training workshops (Appendix). The

questions refer to:

a)The workshop and the lecturers as regards:

- TA objectives (Q1.The objectives for the session were clearly explained).

- The interaction between lecturers and participants workshop (Q2.The instructor

raised questions and posed problems for workshop participants; Q3.The

____________________________________________________________________________________



lecturer was sensitive to the participants’ interests, priorities, and concerns;

Q4.There was a genuine effort to get participants involved in discussions about

the use of VISIR).

- The time allotted (Q5.The time allotted for presentation and discussions was

enough).

b)The use of technological equipment, i.e. VISIR Lab, as regards:

- didactic implications (Q6. The technological equipment enhanced the

effectiveness of teaching and learning).

- practical use (Q8. How difficult do you feel about the practice for VISIR?)

c) Participants’ expectations (Q7. Overall, the presentation about the VISIR

systems met my expectations)

All questions had an evaluation range from 1 to 5 which expressed:

a. participant’s evaluation of the workshop aspect for Q1 to Q6 with the following

option: 1.Unsatisfactory; 2. below average;3. Average; 4. Above average; 5.

Excellent.

b. participant’s evaluation of level of achievement for Q7 with the following options:

1.Poor , 2. Fair, 3. Satisfactory, 4. Highly satisfactory, 5. Excellent.

c. participant’s evaluation of range of difficulty for Q8 with the following options: 1.

Too difficult, 2. Difficult, 3. Just right, 4. Easy, 5.Too easy.

The option NA (0), not answered was also included.

An open question was also included in the satisfaction questionnaire in order to provide

a qualitative perspective to the evaluation by eliciting reflection on positive and

negative aspects of the whole experience. The questions was: Please write about the

positive aspects of Training Action 2 and the aspects to be improved.

Participants

A number of 124 participants attended the TA2. Most of them (64) were university

professors from host universities, some from Associated Partner Institutions (24). In

Argentina, CONFEDI set up a Dissemination Project by inviting representatives of

____________________________________________________________________________________



engineering universities grouped by geographical regions. Therefore, these participants

attended UNSE and UNR workshops who for the sake of data analysis will be identified

as CONFEDI Participants.

Analysis of Results

From a grand total of 87 questionnaires administered, 31 were answered at UFSC, 8 at

IFSC, 7 at PUCRio, 22 at UNSE and 19 at UNR. Chart 1 below shows the results.

Country Brazil Argentina

Grand

total Institution UFSC IFSC PUC Rio UNSE UNR

Total per institution 31 8 7 22 19 87

Chart 1. Number of SQ answers

From 87 questionnaires, 696 questions were analyzed. Only 7 questions were not

answered which represents 1% from grand total. Results from Open Questions were

71 answers, ie, 81,6% total of the survey, and provided very rich information.

The results are shown below in charts and figures. Charts show the scoring given for

each each in each institution; the last line sums up the results in percentages. Figures

present the information in two axes. Each horizontal bar represents in the X axis both:

1. the extent to which evaluation by participants have chosen between the rank from 1

to 5 and 2. the number of answers given according to the number of participants in

each session. Each colour stands for each institution: UFSC blue, IFSC, red, PUC.Rio

green, UNSE purple and UNR light blue. The Y axis represents the rank of evaluation

from 1 to 5. Detailed institutional results are in the Appendix.

Question 1. Objectives

The first question “The objectives for the session were clearly explained” refers to three

dimensions of objectives which all TA2 had to address in one way or another.

____________________________________________________________________________________



1. The objectives of the VISIR+ Project which would provide the contextualization of

the workshops. They were:

The VISIR+ project objectives are: (O1) to allow teachers enriching course curricula, on

electric and electronic circuits theory and practice, by including hands-on, simulated and

remote labs; (O2) to scaffold student’s learning and foster their autonomy, namely by

allowing them to conduct real experiments, over the Internet (on a 24/7 basis); (O3) to

increase students’ meaningful knowledge acquisition and retention by enabling them to

compare results from calculus, simulation and real experiments, at any place / anytime;

(O4) to increase students success rates in continuous assessment modalities,

particularly those covering the acquisition of experimental skills; (O5) and, finally, to

allow the partner institutions using an ICT-based tool for attracting students to STEM

careers, particularly amongst secondary schools

2. The second dimension refers to the objectives of Training Actions 2 in the Project,

narrowing the scope of contextualization:

A 2nd training action in each partner country IHE, led by the two representatives of

each host institution, who attended the 1st training action, plus two representatives of

one European partner . This 2nd action targets all teachers with lecture duties in the

institution’s courses related to electric and electronic circuits, plus two representatives

from the nearby associated partners. Other teachers from the associated partners may

participate remotely. The instructional design of all target courses. At the end of this

activity the expected result is a set of educational modules comprising the use of hands-

on, simulated and remote labs, following an enquiry-based methodology, this

accomplishing O1. The course curricula, lessons plans, and the contents of the target

courses LMS pages will provide the measurable indicators.

3. The third dimension referred to the objectives of each local workshop which implied

getting participants involved in the use of VISIR Labs by understanding the didactic

principles and the technical aspects which underlie remote lab use.

Chart 1 below sums the number of answers under each scoring (1 to 5, including NA

not answered) meaning 1.Unsatisfactory; 2. Below average; 3. Average; 4. Above

average; 5. Excellent.

____________________________________________________________________________________



Host Institution NA 1 2 3 4 5 Total

UFSC - - - 1 11 19 31

IFSC - - - - 2 6 8

PUC Rio - - - - 2 5 7

UNSE 1 - 1 3 10 7 22

UNR - - - - 6 13 19

Total answers 1.14% - 1.14% 5.97% 35.63% 57.47% 87

Chart 1. Objectives

Out of 87 Questionnaires analyzed, 57.47% answered the objectives were explained in

an excellent way, 35.63% considered explanations were above average level and only

5.97% average and 1.14% below average, i.e. only 7.11% seemed to have failed to

understand the objectives of TA2. Only one participant did not answer question 1.

Figure 1 sums up the results of participants’ answers to Q1 in all HE institutions.

____________________________________________________________________________________



Figure 1. Q1. The objectives for the session were clearly explained.

Question 2. Interaction

SQ Question 2 “The instructor raised questions and posed problems for workshop

participants” refers to the didactic approach chosen by lecturers to interact with

participants of workshop. Within the enquiry-based methodological framework which

stands as the pedagogical rationale for the Project, the training sessions aimed at

activating participants’ schemata by meaningful questions and problems rather than

delivering presentations of the teacher-centred lecturing type.

Chart 2 below shows TA2 participants’ answers within the range which was selected:

3. Average; 4. Above average; 5. Excellent.

Host Institution 3 4 5 Total

UFSC 2 7 22 31

IFSC - - 8 8

PUC Rio - 1 6 7

UNSE 4 4 14 22

UNR - 4 15 19

Total answers 6.98% 18.39% 74.71% 87

Chart 2. Interaction

From 87 questionnaires analyzed, 74.71% considered an excellent interaction between

lecturers and participants took place, 18.39% considered the interaction was above

average and only 6.98% average. The figures lead to the conclusion that the content of

presentations was displayed by involving attendees in steady attention and reflection.

Figure 2 below sums up the result in a visual layout.

____________________________________________________________________________________



Figure 2. The instructor raised questions and posed problems for workshop participants.

Question 3. Listening to participants’needs

SQ Question 3 “The lecturer was sensitive to the participants’ interests, priorities, and

concerns” also refers to the didactic factors which contribute to successful

presentations. Interaction implies exchanging ideas, experiences and opinions. Being

sensitive to participants’ needs implies allowing attendees to refer to their own

personal situations in the classroom, their own teaching of content subjects related to

electric and electronic circuits and their stance as to problems which usually turn up

when carrying out lab practice.

Chart 3 below also shows the results in the range 3 to 5 where 3. Average; 4. Above

average; 5. Excellent. Only 2.98% from 87 answers reveal lecturers’ were average in

their concert with participants’ needs while almost 70% consider they were excellent

and almost 30% they were above average.

____________________________________________________________________________________



Host Institutions 3 4 5 Total

UFSC 1 8 22 31

IFSC - 2 6 8

PUC Rio - 1 6 7

UNSE 1 10 11 22

UNR - 4 15 19

Total answers 2.98% 28.73

% 68.96% 87

Chart 3. Participants’ needs

Figure 3 below show the results in all HE institutions.

Figure 3. The lecturer was sensitive to the participants’ interests, priorities, and concerns.

Question 4. Enhancing motivation

Satisfaction Questionnaire Question 4 “There was a genuine effort to get participants

involved in discussions about the use of VISIR” is also part of the group of questions

which intends to explore the didactic resources used to carry out the presentations.

Motivation about VISIR Lab, which for most participants proved a new teaching-

learning technique, was a challenge for TA2. Participants’ interest about the many

aspects which contribute to make remote labs such resourceful tool plus the fact that

____________________________________________________________________________________



there was a main focus on electric circuits became paramount to TA2 effectiveness.

Also, advantages and disadvantages of VISIR remote lab had to be aired in order to

give participants a clear view of what using VISIR implies. It was at this point that

lecturers’ expertise on the use of remote labs for their own university classes turned

out to be essential to engage participants’ interest.

Chart 4 below show the results: from 87 answers, 63.21% considered lecturers

managed to get participants involved in an excellent manner; 33.33% above average

and only 2.29% and 1.14% in an average or below average way. Almost 97% agreed

the level of engagement about VISIR Lab was high.

Host Institution 2 3 4 5 Total

UFSC - 1 9 21 31

IFSC - - 1 7 8

PUC Rio - - 1 6 7

UNSE 1 1 10 10 22

UNR - - 8 11 19

Total answers 1.14% 2.29% 33.33% 63.21% 87

Chart 4. Participants’ interest on VISIR

Figure 4 below shows the results in two axes.

____________________________________________________________________________________



Figure 4. Effort to get participants involved in discussions about the use of VISIR

Question 5. Timing

The time allotted to TA2 varied in each institution, as reported in “TA2 General

Report“ “The time load of agendas varied in every session. They ran from two full

morning shifts (IFSC y UFSC), plus a post lunch slot in PUC Rio, to morning and

afternoon shifts of five modules (UNR) and eight modules (UNSE)”. However, all

agendas fulfilled the minuum time load of 6 hours presented in the “TA2 General

Guidelines”.

Chart 5 below show the participants’ answers to the question “The time allotted for

presentation and discussions was enough” in a scoring from 2 to 5, meaning Below

average, average, above average and excellent.

____________________________________________________________________________________



Host Institution NA 1 2 3 4 5 Total

UFSC 1 - - 9 9 12 31

IFSC - - - - 2 6 8

PUC Rio - - - - 2 5 9

UNSE 2 - - 8 7 5 22

UNR - - - 5 12 2 19

Total answers 3.44% - - 25.28% 36.78 34.48% 87

Chart 5. Timing

From 87 questions, 3 were not answered. All answers show even results as to

34.48% considered an excellent time load, 36.78% above average and 25.28%

average.

Results may suggest participants could have needed more time to come to grips

with VISIR Lab, especially larger groups as in the case of UFSC, UNSE and to a

lesser extent UNR. Figure 5 below shows this tendency.

Figure 5. The time allotted for presentation and discussions was enough.

____________________________________________________________________________________



Question 6. Equipment

Question 6 in Satisfaction Questionnaires evaluated participants’ opinions as to

whether “the technological equipment enhanced the effectiveness of teaching and

learning”. Answers ranged mainly from 3 to 5, i.e. average, above average and

excellent. Chart 6 below shows even percentages being the highest for above average

with 39%, second excellent with 34.48% and third average with 20.68% . Only 4.59%

find equipment below average at to enhancing teaching-learning effectiveness.

Host Institutions NA 1 2 3 4 5 Total

UFSC 1 - 1 4 15 10 31

IFSC - - - - 2 6 8

PUC Rio - - - 1 2 4 7

UNSE - - 3 9 6 4 22

UNR - - - 4 9 6 19

Total answers 1.14% - 4.59% 20.68% 39% 34.48% 87

Chart 6. Equipment effectiveness

It is worth mentioning that at the moment TA2 took place, only one HE Partner

institution had their VISIT Lab installed. All other institutions resorted to the equipment

they have in their European institutions. Another important aspect related to equipment

effectiveness is Internet connectivity which many a time could prove a handicap for

short term training sessions.

Figure 6 below shows answers to Q6. The distribution of answers in the four horizontal

bars reveal that the Lab installation seemed not to be a determinant factor to

participants’ opinions.

____________________________________________________________________________________



Figure 6. The technological equipment

Question 8. User-friendly Remote Lab

Question 8 was also connected to equipment evaluation. The question How difficult do

you feel about the practice for VISIR? refers mainly to the actual interface

characteristics of the Remote Lab which enable the different lab activities as e.g.

measurements.

Chart 7 below shows the answers. Above half of the answers (58.62%) consider “just

right” (3) while a low 4.59 “too easy” and 29.88% “easy”. 5.74% found practice with

VISIR “difficult”.

Host answers NA 1 2 3 4 5 Total UFSC 1 - 3 19 7 1 31 IFSC - - - 2 5 1 8 PUC Rio - - - 3 3 1 7 UNSE - - 2 14 6 0 22 UNR - - - 13 5 1 19 Total answers 1.14% - 5.74% 58.62% 29.88% 4.59% 87

Chart 7. VISIR Practice

____________________________________________________________________________________



Figure 7 clearly shows the highest percentage of participants’ answers considered

were VISIR practice neither difficult nor easy. The distribution of answers among HE

institutions is even.

Figure 7. How difficult do you feel about the practice for VISIR?

Question 7. Participants’ expectations about VISIR

Finally, Question 7 intended to give participants the opportunity of evaluating the

overall TA2 perfomance. The questions was “Overall, the presentation about the VISIR

systems met my expectations” and the options ranged from 1.Poor, 2. Fair, 3.

Satisfactory, 4. Highly satisfactory, 5. Excellent.

____________________________________________________________________________________



Host Institutions 2 3 4 5 Total

UFSC 1 2 13 15 31

IFSC 0 3 2 3 8

PUC Rio - 1 1 5 7

UNSE 6 6 6 4 22

UNR - 2 8 9 19

Total answers 8.04% 16.09% 34.48% 41.37% 87

Chart 8. TA2 overall impression

From 87 questions analyzed, 41.35% participants considered the training actions were

excellent in meeting their expectations and 34.48% were highly satisfactory. Only

16.09% found the workshop satisfactory and 8.04% fair.

Figure 8 shows the results in two axes per institution.

Figure 8. The presentation about the VISIR systems

____________________________________________________________________________________



Qualitative Analysis: Positive aspects and aspects to be improved in TA2

The aim of open question number 9 aimed at eliciting qualitative information about

positive and negative aspects of training actions. Although explicit encouragement and

plenty of room (both, time and questionnaire layout) was given for participants to write

down airing their opinions in paper, few attended answered this questions and those

who did write short comments.

On the positive aspects there is reference to the learning environment as regards

lecturers’ assets (“kindness”, “clarity”, “feedback”) and their presentations (“I could

understand information about VISIR and how to use it”; “Visual presentations were very

effective”). Also some positive comments refer to the value of VISIR Lab as a tool (“the

potential usefulness of VISIR could be observed”. The TA organization and the

possibility of attending to them was also pointed out.

As to the aspects to be improved, most comments refer to the need to count on more

time availability to practise the use of the remote lab, to exchange experiences and to

explore the possibilities VISIR has. Wifi connections was also highlighted as a key

aspect to facilitate or hinder lab use (“There was saturation in wifi connection making

the online use slow”).

Finally some recommendations for extension of the experience were given: “I hope

VISIR could be taken to Angola, my country” and “The lab has to be promoted to many

departments of electric engineering careers”.

Conclusion

The information presented from Satisfaction Questionnaires applied to participants of

the five training actions which took place in Latin America HE Institution, reveal that the

workshops have fulfilled their aims: the objectives of the workshops were clearly

presented and the learning environment created was effective to engage participants in

the possibility of using VISIR lab in their engineering classes.

____________________________________________________________________________________



The challenge ahead in the implementation in actual engineering courses and the

design of Modules which could lend themselves to research experiences and the

rendering of knowledge about the use of VISIR Remote Labs.

____________________________________________________________________________________



Appendix: Detailed institutional results (in alphabetical order)

Question 1

____________________________________________________________________________________



____________________________________________________________________________________



____________________________________________________________________________________



Question 2

____________________________________________________________________________________



____________________________________________________________________________________



____________________________________________________________________________________



Question 3

____________________________________________________________________________________



____________________________________________________________________________________



____________________________________________________________________________________



Question 4

____________________________________________________________________________________



____________________________________________________________________________________



____________________________________________________________________________________



Question 5

____________________________________________________________________________________



____________________________________________________________________________________



____________________________________________________________________________________



Question 6

____________________________________________________________________________________



____________________________________________________________________________________



____________________________________________________________________________________



Question 7

____________________________________________________________________________________



____________________________________________________________________________________



____________________________________________________________________________________



Question 8

____________________________________________________________________________________



____________________________________________________________________________________



Documents

TRAINING ACTION Nº 2 SATISFACTION … General Report.pdf · Higher-Education Partners from ... 561735-EPP-1-2015-1 ... Figure 1 sums up the results of participants’ answers to