ClipFlair Pilot Use Report

CLIPFLAIR Foreign Language Learning Through Interactive Revoicing and Captioning of Clips

LIFELONG LEARNING PROGRAMME - KEY ACTIVITY 2 LANGUAGES, MULTILATERAL PROJECT

519085-LLP-1-2011-1-ES-KA2-KA2MP

D5.1

ClipFlair Pilot Use Report

WP No. WP5

WP Title Pilot use of the produced activities by FL learners (final)

Activity description

This deliverable records the findings from the final stage of the pilot use of the project outputs (academic year 2013-14). They will be used as source material for the final evaluation report. End users: project members

Authors Universitat Autònoma de Barcelona (UAB): Lupe Romero, Helena Casas-Tost, Anabel Galán-Mañas, Lucía Molina, Patricia Rodríguez-Inés, Sara Rovira, Olga Torres-Hostench

Status (D: draft; RD: revised draft; F: final) F

File Name D5.1.PilotUseReport.doc

Date 15 July 2014

July 2014 Page | 1

D5.1b. ClipFlair Pilot Use Report (final stage)

EXECUTIVE SUMMARY

This deliverable records the findings from the final stage of the pilot use of the project outputs (academic year 2013-14). They will be used as source material for the final evaluation report. This document includes:

• Details on the organization of the final pilot test (general information on the test, sample information, questionnaires, etc.)

• Teachers’ results

• Students’ results

• Conclusions from the pilot test and suggestions for improvement

• Results by universities

July 2014 Page | 2

D5.1b. ClipFlair Pilot Use Report

Table of Contents

1 ORGANIZATION OF THE PILOT USE OF THE PLATFORM .............................................. 7

1.1 The questionnaire for the teacher ............................................................................................. 8

1.2 The questionnaire for the student ............................................................................................. 8

2 RESULTS .............................................................................................................. 10

2.1 Description ............................................................................................................................ 10

2.1.1 Universities ................................................................................................................... 10

2.1.2 Teachers........................................................................................................................ 11

2.1.3 Students ........................................................................................................................ 11

2.1.4 Languages ..................................................................................................................... 12

2.1.5 Activities ....................................................................................................................... 12

2.2 Teachers’ Results ................................................................................................................... 12

2.2.1 Results regarding the activities ........................................................................................ 12

2.2.2 Results regarding the technical problems ......................................................................... 15

2.3 Students’ results .................................................................................................................... 16

2.3.1 Overall results ................................................................................................................ 16

2.3.2 Students’ Profile ............................................................................................................ 16

2.3.3 About ClipFlair activities ................................................................................................. 25

2.3.4 About ClipFlair Studio ..................................................................................................... 26

2.3.5 About the ClipFlair web application ................................................................................. 27

2.3.6 About the ClipFlair social network ................................................................................... 28

2.4 Pilot use by university ............................................................................................................ 32

2.4.1 Universitat Autònoma de Barcelona (UAB) ....................................................................... 32

July 2014 Page | 3


2.4.2 Universitat Pompeu Fabra (UPF) ..................................................................................... 36

2.4.3 Imperial College London (ICL) and University College London (UCL) ................................... 41

2.4.4 Universitatea „Babeș-Bolyai”, Cluj-Napoca, Romania (UBB) ............................................... 44

2.4.5 University of Deusto ....................................................................................................... 49

2.4.6 University of Tallinn ....................................................................................................... 52

2.4.7 University of Warsaw ..................................................................................................... 55

2.4.8 National University of Ireland, Galway (NUI Galway) ......................................................... 61

3 COMPARISON BETWEEN PILOT PHASES .................................................................. 65

3.1 Participating Institutions ........................................................................................................ 65

3.2 Participants ........................................................................................................................... 65

3.3 Activities ................................................................................................................................ 65

3.4 Teachers' results .................................................................................................................... 66

3.5 Students' results .................................................................................................................... 66

APPENDIX 1. Teachers’ questionnaire .................................................................................................. 67

APPENDIX 2. Students’ questionnaire ................................................................................................... 70

APPENDIX 3. Pictures from piloted activities......................................................................................... 74

APPENDIX 4: Reminder sent to students via email ................................................................................ 75

List of Tables

Table 1. Summary of piloted activities…………………………………………………………………………………12

Table 2. Summary of UAB team work……………………………………………………………………………35

Table 3. Summary of teachers involved in the piloting at UAB………………………………………..36

July 2014 Page | 4


List of Figures

Figure 1. Where did students do the activity? ............................................................. 13

Figure 2. Where did students perform the activity? [Source: Deliverable 1136 (first pilot

test)] 14

Figure 3. How often teachers use audiovisual materials ............................................... 14

Figure 4. How often do you use other kinds of audiovisual activities in this course?

[Source: Deliverable 1136 (first pilot test)] ..................................................................... 15

Figure 5. Students’ native languages .......................................................................... 17

Figure 6. Students’ native languages [Source: Deliverable 1136 (first pilot test)]............. 17

Figure 7. Students’ gender ........................................................................................ 18

Figure 8. Students’ age group .................................................................................... 18

Figure 9. Students’ current place of study ................................................................... 19

Figure 10. Students’ current place of study [Source: Deliverable 1136 (first pilot test)] .. 19

Figure 11. Completed studies ................................................................................... 20

Figure 12. Completed studies [Source: Deliverable 1136 (first pilot test)] ..................... 20

Figure 13. Activity language proficiency level ............................................................. 21

Figure 14. Previous use of multimedia in language lessons ......................................... 22

Figure 15. Like/Dislike the use of audiovisual material ................................................ 22

Figure 16. Like/Dislike working with computers ......................................................... 23

Figure 17. Reasons for not having completed the activity ........................................... 24

Figure 18. Usefulness of ClipFlair activity for language learning ................................... 25

Figure 19. Technical problems .................................................................................. 26

July 2014 Page | 5


Figure 20. Main types of technical problems ............................................................. 27

Figure 21. About the ClipFlair web application ........................................................... 28

Figure 22. Use of ClipFlair social network .................................................................. 28

Figure 23. Piloting at IES Esteve Terradas .................................................................. 34

Figure 24. Use of ClipFlair social network .................................................................. 35

July 2014 Page | 6


1 ORGANIZATION OF THE PILOT USE OF THE PLATFORM

The UAB partner established the procedure for the final pilot test in coordination with the Project Managers at UPF. The procedure consisted of the following steps:

• Developing the instructions to fill out the questionnaires. • Controlling the development of the pilot phase. • Analysing the results from the questionnaires.

The pilot procedure was shared with the rest of project partners. Questions and comments regarding the pilot phase were posted and answered in the project management tool. The instructions to be followed by all partners during the pilot phase were the following:

1. All partners should use the following forms after the piloting of each activity. Both forms are also available in the Documents area of Redmine.

a. Learner feedback form: https://docs.google.com/forms/d/1oqBPyNDwVBEK33MwmSeNXEiBthjG5UdYHwJLFTUphmo/viewform

b. Pilot test information form to be filled in by the teacher: https://docs.google.com/forms/d/1km6wmBgQczBehgZYyLhUGeXshF21Ekx-8waeDDW3ACE/viewform

3. The deadline to pilot the activities and to submit both questionnaires is 17th May 2014.

4. You might need to translate the learner feedback form. In that case we will make a copy of the form and give you rights to edit it and translate it. Then we will merge all results in the same Excel file.

5. The UAB team will collect all the data from the submitted questionnaires and will post it in Redmine as an Excel file.

6. All partners should check their data in Redmine and translate into English those answers in languages different from English before 30th May.

7. Partners are kindly asked to post on Redmine team pictures from their piloting experience.

July 2014 Page | 7

https://docs.google.com/forms/d/1oqBPyNDwVBEK33MwmSeNXEiBthjG5UdYHwJLFTUphmo/viewform



https://docs.google.com/forms/d/1km6wmBgQczBehgZYyLhUGeXshF21Ekx-8waeDDW3ACE/viewform

https://docs.google.com/forms/d/1km6wmBgQczBehgZYyLhUGeXshF21Ekx-8waeDDW3ACE/viewform


8. The UAB team will write the final pilot test report to be discussed in Barcelona’s final meeting (June 2014).

The UAB Team encouraged project partners to participate in the pilot phase and, in due time, reminded them of the deadline. Results from students and teachers participating in the pilot phase were recorded in a GoogleDocs file. After deadline on 17th May the UAB team started analyzing the results in order to prepare this report.

It has to be noted that not all users of the platform filled in the questionnaire. The complete and detailed pilot use by partner can be seen in section “ Pilot use by university” page 32

1.1 The questionnaire for the teacher

A form to be filled in by every teacher after piloting one of his/her own activities was originally designed by the UAB team and later agreed upon with several other partners. The form consisted of three blocks (see Appendix 1).

The first block included teacher’s data, such as name and surname, e-mail address and centre affiliation.

The second block focused on activity data, such as the title of the piloted activity (identified by the URL link) and a question about the time students needed to complete the activity.

The last block aimed to gather detailed information about the context where the activity was piloted, such as the course, the learning environment (in class with the teacher, in a lab without teacher supervision, or at home), number of students, how often the teacher uses other types of audiovisual activities in his/her course).

A box was available for teachers to comment on any technical problems experienced or any other aspect that could be useful as feedback.

1.2 The questionnaire for the student

The questionnaire to be filled in by students is divided into four blocks (see Appendix 2).

The first one, Pilot study data, collects three basic data:

July 2014 Page | 8


(1) the centre in which the activity was performed;

(2) the name of the teacher who piloted the activity;

(3) whether the student completed the activity or not. A negative answer to the last question would not allow the student to fill in the questionnaire of the activity, but led him/her to a window with possible reasons for not having finished the activity. The rationale behind these items (I didn't have time, It was too complicated, I had technical problems, I did not like the video, I don't like computers, I didn't understand what I had to do, I don't feel comfortable recording my voice) was to provide data on both the activity and the platform.

The second block, About the ClipFlair Activity, aims to describe the learner experience with the activity. Therefore, the most important item within this part is “The activity was…interesting, fun, useful for language learning, useful for improving my competence in translation, easy, had clear instructions, I would like to do more activities like the one I did”. Students choose their answers from an ordinary scale (Not at all, No, More or less, Yes, Very much). This block also includes an item regarding technical problems.

The third block, About the ClipFlair Studio, seeks to show the learner experience with the web application. It consists of two questions, one related to the platform and the other to the social network. The first question crosses opinions (The ClipFlair Studio is user-friendly, The interface is attractive, On the whole, I enjoyed the ClipFlair experience) with an ordinary scale (Not at all, No, More or less, Yes, Very much). The second question is “Did you use the ClipFlair Social Network?” and the possible answers are Yes or Not. At the end of this part students can also add their comments.

The fourth and last block is About the learner. It collects personal data (country of origin, native language, gender, age, centre where he/she is studying, his/her proficiency level in the language of the activity), and data about the learner’s experience with audiovisual materials (I like working with clips and audiovisual material to learn foreign languages, I like working with computers).

July 2014 Page | 9


2 RESULTS

2.1 Description

2.1.1 Universities

A total of 20 academic centres have participated in both pilot tests: 7 ClipFlair Partners and 13 associate partners. From those, 13 are universities and 7 non-university centres. In first pilot test, 8 universities participated: 6 ClipFlair partners and 2 associate partners (also universities).

Partner Universities Associate partners

ClipFlair P3. UAB, Barcelona (Spain)

ClipFlair P5. UBBCLUJ, Cluj (Romania)

ClipFlair P7. TLU, Tallinn (Estonia)

ClipFlair P10. NUI, Galway (Ireland)

ClipFlair. Algarve (Portugal)

ClipFlair. Deusto (Spain)

ClipFlair. UCL (UK)

UNED, Madrid (Spain)

HOU, Athens (Greece)

Polygyros School (Greece)

University of Leeds (UK)

IES Esteve Terradas

Kurutziaga Ikastola

Lawdale Primary School

Private Lessons at Worksite

Warsaw University of Technology

University of Pavia

Mercy College, Sligo

Kazimierz Wielki University

EUROPASS Centro Studi Europeo

July 2014 Page | 10


2.1.2 Teachers

A total of 37 different teachers have taken part in the piloting of activities (12 in the first pilot test). Some of these teachers have piloted activities with groups from other teachers, disseminating the platform to teachers not involved in the project. The rest of teachers are active members of the ClipFlair project.

2.1.3 Students

A total of 1219 student questionnaires (318 in the first pilot test) were answered by students during the pilot phase. A total of 84 different ClipFlair activities were done. There are 23 questionnaires that cannot be related to a particular activity due to missing data.

Centre Learners Teachers

Universitat Autònoma de Barcelona, Spain 433 9

University Babes Bolyai, Romania 394 7

Tallinn University, Estonia 38 3

National University of Ireland, Galway 54 6

Imperial College London, UK 21 2

University of Leeds, UK 11 1

Universidad Nacional de Educación a Distancia, Spain

41 1

Hellenic Open University, Greece 24 1

First Primary School of Polygyros, Greece 18 1

University College London, UK 7 1

Universidad de Deusto, Spain 11 1

University of Algarve, Portugal 16 1

July 2014 Page | 11


Universitat Pompeu Fabra, Spain 44 2

Other Secondary Education and Adult Learning Institutions: (Institut Esteve Terradas, Warsaw University of Technology, Università di Pavia, Europass Centro Studi Europeo, Mercy College, Kurutziaga Ikastola)

107 5

TOTAL 1,219 41

Table 1. Summary of completed questionnaires

2.1.4 Languages

Twelve different languages have been used in the final pilot test (as opposed to 7 languages in the first pilot test): English, Portuguese, Spanish, Arabic, Chinese, Catalan, Romanian, Polish, Basque, Irish, Estonian and Italian. 20.7% of activities in the ClipFlair Gallery were designed to practice English and around 13.8% of activities to practice non-EU languages: Chinese, Japanese and Arabic. From the questionnaires provided by teachers we know that there is a small proportion of activities (13%) designed to practice translation with different language combination.

2.1.5 Activities

In total, 84 activities have been piloted.

2.2 Teachers’ Results

This section presents the results collected using the teacher’s feedback form known as Pilot use information (see Appendix 1). This form had to be filled in by every teacher after piloting one of his/her own activities. It includes the results received until the 17th May, which was the deadline for the activity pilot phase. Below is a summary of where the students carried out the piloted activity, how often the teacher used other kinds of audiovisual activities in his/her course, and a classification by categories of the technical problems reported by teachers. A total of 25 teachers answered the teacher’s form.

2.2.1 Results regarding the activities

Where did students perform the activity?

July 2014 Page | 12


Most activities were piloted in class together with the teacher; only 30% of them were piloted without teacher supervision, either in a language lab or at home.

Some of the activities were done in class with the teacher, but students did not have enough time to fill in the feedback questionnaire and were asked to do it at home. This may be the reason why not all students filled in the questionnaire.

Figure 1. Where did students do the activity?

In order to be able to compare total results with first pilot results, see below the results from the first pilot test:

70%

24%

6%

Where did students do the activity?

In class with the teacher

At home

In a lab without teachersupervision

July 2014 Page | 13


Figure 2. Where did students perform the activity? [Source:

Deliverable 1136 (first pilot test)]

Length of activity

The length of activities ranged from less than 5 minutes to more than 30 minutes, except for two activities that were done at home and the teacher did not know how long students took to do it.

How often do you use other kinds of audiovisual activities in this course?

Only around 25% of the teachers involved in the piloting phase use audiovisual materials often or very often, while most use them every now and then or seldom.

Figure 3. How often teachers use audiovisual materials

Notice that some teachers that never use audiovisual materials (14%) and have decided to use ClipFlair in the second pilot test, which is good news. In the first pilot test, more teachers were used to audiovisual materials in class (see below):

43%

19%

14%

13%

11%

How often teachers use audiovisual materials

Seldom (once every twomonths or less)

Now and then (onceevery 2-4 weeks)

Never

Often (once a week)

Very often (more thanonce a week)

July 2014 Page | 14


Figure 4. How often do you use other kinds of audiovisual activities in this course? [Source: Deliverable 1136 (first pilot test)]

2.2.2 Results regarding the technical problems As far as technical problems experienced during the second pilot phase is concerned, teachers reported technical problems in only 23% of the cases, slightly lower than in the preliminary pilot test (26%). Most technical problems during second pilot test were not due to ClipFlair but to other causes, such as:

“Some students had problems with Silverlight”

“A bit of buffering at the video, but the internet connexion was slow”

“Students experienced problems using University PCs (Windows xp) and their laptops (Windows 7 and 8)”

Concerning audio, teachers reported the following technical problems:

“One revoicing was extremely low (mic was working properly), another one recorded student's voice over original audio without and one could hear both voices. In another case, sentences were clipped (recordings seemed to be incomplete).”

“Initially, but due to the way the sound was organized in that particular lab. Once the proper programmes were activate, everything worked very well.”

“They had problems trying to record their voices”

July 2014 Page | 15


“Students complained about the fact that they cannot see how long their recordings are. In addition, when students pause the video to listen to their recordings, their recordings keep going on and cannot be paused.”

“Revoicing didn't work properly. Students' voices were mixed with the original soundtrack so that both English and Italian audio could be heard.”

Concerning captions, one teacher reported that “*It'd be good if the platform allowed the teacher to set a maximum number of characters allowed per subtitle”, and actually, the program allows to mark a maximum number of characters.

It is worth mentioning that in Ireland, for the ClipFlair website to become accessible to secondary school, a request had to be sent to the National Teachers’ Council. The petition was sent and accepted.

2.3 Students’ results

2.3.1 Overall results

In this section the results of the questionnaire filled out by students are presented: results about students’ profile; results about ClipFlair activities; results about ClipFlair Studio and results about the ClipFlair web application.

2.3.2 Students’ Profile

These data have been obtained from the Students’ Questionnaire (see Appendix 2).

Although 1173 students participated in the pilot tests, only 1076 completed a ClipFlair

activity. Thirty-seven teachers coming from 13 different universities were involved in

the pilot studies: National University of Ireland (Galway), Universitat Autònoma de

Barcelona and Universitatea Babeş-Bolyai, Tallinn University, Hellenic Open

University, Algarve, Deusto, UCL, HOU, University of Leeds and UNED, plus non-

university centres such as Polygyros, IES Esteve Terradas, Kurutziaga Ikastola,

Lawdale Primary School, and Private Lessons at Worksite.

The students participating in the pilot tests came from 30 different countries. Most of

them come from European countries (98%), although there are respondents from

Africa, America, Asia and Australia. Among all the students, 65% said they have a

July 2014 Page | 16


single mother tongue; 16% considered him/herself bilingual and 19% trilingual.

Compared to the preliminary pilot test, many more students considered themselves

to be bilingual and trilingual.

Figure 5. Students’ native languages

Notice differences between final results and first pilot test below:

Figure 6. Students’ native languages [Source: Deliverable 1136

(first pilot test)]

Regarding gender, the vast majority, 75%, are women, just 3% less than in the

preliminary pilot test.

65%

16%

19%

Native speaker of

1 language

2 languages

3 languages

89%

9% 2% Native speaker of

Onelanguage

July 2014 Page | 17


Figure 7. Students’ gender

Regarding age, the vast majority of students, 74%, belong to the age group between

18 and 35. The group aged between 13 and 18 represents 13%, the group aged

under 13 represents 10% and the group aged over 35 represents the remaining 3%.

Compared to the first pilot test, there are considerable less students in the age group

between 18 and 35 (from 93% to 74%) and there has been an increase in the age

groups below 18 years old, while the group above 35 has remained more or less

steady.

Figure 8. Students’ age group

75%

25%

Gender

Female

Male

74%

13%

10%

3%

Age group

18 - 35 years old

13 - 18 years old

< 13 years old

> 35 years old

July 2014 Page | 18


In the item “I am a student at”, the answer "University" was the most popular (77%),

which is probably the case because most of the activities were piloted by teachers

from universities involved in the ClipFlair project.

Figure 9. Students’ current place of study

These results can be compared with those of first pilot test:

Figure 10. Students’ current place of study [Source: Deliverable 1136 (first pilot test)]

77%

18%

4% 1%

I am a student at

University

Secondary School

Other

Language School

0,35 0,35

99,31

Student of

Languageschool

Secondaryschool

University

July 2014 Page | 19


The fact that almost all the students who participated in the pilot test were currently

pursuing studies at university conditions the results from another item: “Level of

Studies completed”. More than half of the students, 60%, answered “school”, while

another significant percentage of the students, 27%, stated they hold a university

degree. We assume that they are currently pursuing their second or third university

degree.

Figure 11. Completed studies

These results can be compared with those of first pilot test:

Figure 12. Completed studies [Source: Deliverable 1136 (first pilot

test)]

60% 27%

8%

3% 2%

Level of studies

School

University degree

Other

Post-graduate (MA, PhD)

Vocational Training

37%

4% 3%

48%

8%

Level of studies

Universitydegree

Post-graduate(MA, PhD)VocationalTraining

July 2014 Page | 20


The item “Proficiency level in the language of the activity” shows that most

participating students rated their knowledge of the language as upper intermediate

(4) or intermediate (3).

Figure 13. Activity language proficiency level

The questionnaire includes three questions inquiring about the students’ familiarity

with using audiovisual material and interest in the technical and pedagogical

environment of ClipFlair activities: computer and clips.

It should be noted that nearly a quarter of the students rated “I have used video or

multimedia as a language student before doing any ClipFlair Activity” with the lowest

score.

10%

14%

26% 28%

22%

Proficiency level in the language of the activity

1

2

3

4

5

July 2014 Page | 21


Figure 14. Previous use of multimedia in language lessons

In contrast, 75% of the students responded very positively to the statement “I like

working with clips and audiovisual material to learn foreign languages”, scoring 4 or 5

out of 5.

Figure 15. Like/Dislike the use of audiovisual material

22%

15%

29%

21%

13%

I have used video or multimedia as a language student before doing any

ClipFlair Activity

1

2

3

4

5

3%

5%

17%

33%

42%

I like working with clips and audiovisual material to learn foreign

languages

1

2

3

4

5

July 2014 Page | 22


The same positive result can be observed in relation to the last statement, “I like

working with computers”.

Figure 16. Like/Dislike working with computers

Finally, the reasons provided by students who did not finish their activity are

displayed below. It can be observed that the vast majority, 76%, argues lack of time.

We think that this “lack of time” is probably related to the novelty that the ClipFlair

environment represents for the student. Probably, if the student were familiar with the

platform, he/she would have been faster.

3%

5%

15%

33%

44%

I like working with computers

1

2

3

4

5

July 2014 Page | 23


Figure 17. Reasons for not having completed the activity

76%

6%

2% 2%

2%

2% 2%

2% 2%

1% 1%

1%

1%

0%

I did not complete the ClipFlair activity because

I didn't have time

I had technicalproblems

It was too complicated,I didn't understandwhat I had to do

I didn't have time, Idon't feel comfortablerecording my voice

It was too complicated

I didn't have time, Ihad technical problems

It was too complicated,I didn't understandwhat I had to do, Idon't feel comfortablerecording my voiceI didn't understandwhat I had to do

July 2014 Page | 24


2.3.3 About ClipFlair activities

Over 80% of students answered “yes” or “very much” to the statements “The activity was useful for language learning” and “The activity was useful for improving my competence in translation”. In addition, over 60% considered that the instructions were clear. According to this, it can be said that the answers that got best results are those related to the pedagogical usefulness of ClipFlair activities.

Figure 18. Usefulness of ClipFlair activity for language learning

0

100

200

300

400

500

600

Yes

Yes, very much so

More or less

No

Not at all

July 2014 Page | 25


2.3.4 About ClipFlair Studio

Over 80% of students had no technical difficulties in the second phase of the piloting of the platform as shown in the questionnaires filled in by students during 2014. Even though the figures are similar to those in the first phase, most of the problems pointed out in the interim report have already been solved. It is important to highlight that some technical problems were experienced by just one user and were related to rather personal circumstances (e.g. their PC at home, their Internet connection).

Figure 19. Technical problems

It is to be highlighted that several problems mentioned in the interim report have been solved satisfactorily and are not mentioned anymore. These problems are the loading of videos and compatibility of the system with the computer, as well as the fact that, in captions, it was not possible to write from right to left, as in Arabic. Additionally, 13 users said that they had experienced technical problems but never specified of what type.

83%

17%

I had technical problems

No

Yes (please specify in thenext question)

July 2014 Page | 26


Figure 20. Main types of technical problems

2.3.5 About the ClipFlair web application

Generally speaking, the three web application-related items included in the questionnaire have had positive answers. In the first item, “The ClipFlair Studio is user-friendly”, almost 80% of students have answered “yes” or “very much so”. In the second, “The interface is attractive”, the response has been less enthusiastic, with “yes” and “more or less” rating almost the same. The third, which is a global question, “On the whole, I enjoyed the ClipFlair experience” has been rated highest if we take into account both answers “yes” and “very much so”.

28%

14% 35%

23%

Main types of technical problems The audio The captions The synchronisation The video

July 2014 Page | 27


Figure 21. About the ClipFlair web application

2.3.6 About the ClipFlair social network

Figure 18 below shows that almost half of the students (46%) used the ClipFlair social network facilities. Although the percentage is not very high, it increased a lot compared to the first pilot phase, when only 23% of students had used it. Lack of time when doing the activity and answering the questionnaire may be the cause of these results.

Figure 22. Use of ClipFlair social network

0

100

200

300

400

500

600

700

The ClipflairStudio is user-

friendly

The interface isatractive

On the whole, Iejoyed Clipflair

Yes

Yes very much so

More or less

No

Not at all

54%

46%

Clipflair Social Network

No

Yes

July 2014 Page | 28


OBSERVATIONS

Not all students provided observations in their questionnaires. However, some students gave valuable feedback for the partners. Below there is a summary:

Suggestions for improvement from first pilot 2013

Actions taken and situation by the end of project

On the interface: It is a bit unclear Solved It is a bit blurry Solved The resolution of the screen should be improved Non-ClipFlair related The quality of the subtitles on the screen was not very good Solved The zoom should be adjusted somehow because it zooms out of page, and the windows cannot be positioned freely Solved The interface should be more "compact" Solved The recording with the instructions about how to use ClipFlair are repeated, and they could be deleted Activity-related The number of characters should be limited in captions, as in professional subtitling Added feature It could be useful to save the project as a video file

Elements can be saved separately and together

It should be possible to pause the video by pressing the space bar Pause button On technical problems: Problems with recording audio and video Solved Microphone should be checked beforehand Non-ClipFlair related System needs to be adapted to allow for writing from right to left Added feature

Some students also provided feedback to improve the design of activities and the overall learning experience:

July 2014 Page | 29


1) “I think students could benefit with a demo in class before using it at home themselves. After I was shown it all made sense.”

2) “Idea for activity: listening to the video without the captions” 3) Difficulty:” trying to understand the exact words/tenses spoken” 4) Difficulty: “recording the clips” 5) Difficulty: “revoicing the clips in the time allowed was difficult & needed lots of

practice.” 6) Suggestion: ”I think a more detailed explanation of how to set up would help. Our

tutor addressed this issue on our second on campus session and then all was fine.” 7) Suggestion: “ClipFlair should be used to work with vocabulary in a more

intensive manner” 8) Suggestion: “activities should be more interesting”

Positive students’ observations, exactly as they themselves wrote them, are presented below:

POSITIVE OBSERVATIONS (First pilot First semester 2013)

I'm using it for the first time at Chinese class. I find it very interesting, and for being in a beta stage it's very complete: It has so many features that give so many different exercise opportunities to make. I only would make the interface more "compact", but I like it.

It is a fun way of improving your translations' skills.

ClipFlair is an easy to use, yet really useful system. I especially liked the way in which it combines knowledge and learning with fun. I enjoyed using it, since it is something new for me. I will definitely use it again, since I find it interesting.

I think it is a very useful and attractive activity, not only for students but for any speaker of foreign languages who would like to improve his/her knowledge. We all love movies, and the activity is so interesting that it could be done as a hobby too.

I hoped for a bit more interesting exercises.

In my opinion, the exercises which were proposed were very useful for the vocabulary enrichment, so I think it is very useful.

This activity make you to learn different adjectives and practice your pronunciation. It´s very fun and useful for tourism´s students.

This activity was more difficult than the another one about Edinburgh, because here you need to speak about senses and feelings and emotions more than only you need describe the images. But it´s was amazing.

July 2014 Page | 30


In my opinion the web is very well structured.

I think this is a very useful web, given the low number of available webs to learn languages currently

It's an interesting activity and useful for improving your knowledge on this language.

I think the most attractive aspect of this program is how easy it is to use it, and how it helps to discover new things about your language combination.

POSITIVE OBSERVATIONS (Second pilot 2013-2014)

I like the Clipflair application.

I think the platform is a good learning tool.

It was a very interesting and useful experience.

ClipFlair Studio helped especially in the listening/audio section.

I think this is an easy-to-use platform, very useful for students and teachers alike, I enjoyed working with it.

I like watching videos and do practical exercises there. I think the exercises are very useful.

July 2014 Page | 31


2.4 Pilot use by university

2.4.1 Universitat Autònoma de Barcelona (UAB)

Summary of figures related to the Clipflair Project by UAB

56 activities created

25 activities piloted

433 questionnaires

395 questionnaires answered

6 teachers involved in the creation of activities

10 teachers involved in the piloting

Table 2 Summary of UAB team work

Apart from the six UAB teachers who are members of the Clipflair project (who have been in charge of creating and piloting activities), there have been four more teachers who have piloted our activities. Two of these teachers belong to the same institution (UAB) and the remaining two come from the secondary school IES Esteve Terradas. The following table summarizes these figures:

Clipflair members UAB teachers Secondary school

IES. Esteve Terradas

Anabel Galán Maria Regina Dora Garde

July 2014 Page | 32

http://www.esteveterradas.cat/index.php/noticies-ies-125/cicles-formatius/llenguees-estrangeres/1082-el-projecte-clipflair-al-centre




Clipflair members UAB teachers Secondary school

IES. Esteve Terradas

Saraiva Mendes

Helena Casas Tost Xianghong Qu Francesca Ceres

Lucía Molina

Olga Torres Hostench

Patricia Rodríguez-Inés

Sara Rovira-Esteva

Table 3. Teachers involved in the piloting at UAB

IES Esteve Terradas published an entry on the piloting of Clipflair activities in their web page (see snapshot below). For more information click on the following link: http://www.esteveterradas.cat/index.php/noticies-ies-125/cicles-formatius/llenguees-

estrangeres/1082-el-projecte-clipflair-al-centre:

July 2014 Page | 33







Figure 23. Piloting at IES Esteve Terradas

The activities created by the UAB team that have been piloted include the following foreign languages and language combinations, in the case of translation activities:

0

20

40

60

80

100

120

Arabic Chinese Portuguese translationEnglish>Catalan

TranslationEnglish>Spanish

Universitat Autònoma de Barcelona

July 2014 Page | 34


Figure 24. Use of ClipFlair social network

The UAB team has made wide use of the social part of the Clipflair platform. It has resulted in the creation of following groups, together with the amount of users of each one of them:

• Iniciación a la traducción (31)

• TB-A3 (23)

• Traducción científico-técnica FTI (28)

• Tradumatica UNIOR (15)

• Tradumatica (29)

• TAV Chinese-Spanish (UAB) (44)

• TAV Lleida (4)

• Learning Arabic (17)

• Learning Chinese (7)

Finally, the UAB team has contacted different authorities from the Education Department of the Catalan Government (Generalitat de Catalunya), who attended the Clipflair Conference:

• Head of Primary and Secondary school: Neus Lorenzo

• Head of Foreign Language Schools (EOI): Neus Figueres

Below are some pictures of the piloting of activities at the UAB (see more pictures in appendix 3).

July 2014 Page | 35


2.4.2 Universitat Pompeu Fabra (UPF)

UPF Barcelona carried out its piloting activity in a university context but with groups from different undergraduate programs, collaborating in one project.

Summary of figures related to the Clipflair Project by UPF



35 questionnaires




Table 1 Summary of UPF team work

The following UPF teachers have created and piloted the activities:

UPF staff

Elena Voellmer

Rebecca Walter

Table 2. Teachers involved in the piloting at UPF

The activities created by the UPF team that have been piloted include the following foreign languages: English, German, (Spanish and Catalan). The piloting project was carried out April to June 2014. The

July 2014 Page | 36


main questions of the research where; how can ClipFlair be implemented into a university context and iIs its implementation useful for the students. On a theoretical level, the activity tried to look at the competencies it could potentially foster and finally asked for the motivational component and on what students thought about the activity. The particularities of the project were the work with students from different undergraduate studies (Applied Languages and Translation Studies) and having one group translating between two foreign languages (L3 and L4 for most of the students).

The UPF team used the social network of the Clipflair platform during the pilot lessons both for revoicing and captioning. The following groups were created:

Group 1 Group 2

Studies Applied Languages Translation

Group profiles 2nd year, 24 students (in-course obligatory participation)

3rd year, 11 students (voluntary participation)

Language profiles Majority bilingual (ES/CAT) Learn several foreign languages

Majority bilingual (ES/CAT) Learn several foreign languages

Activities -Creating conversation in German (Written to be spoken: modal particles) -Subsequent revoicing (error correction)

Translating German dialogue into English (Captioning)

Competencies used -Plurilingual competence - Multiliteracy competence

-Plurilingual competence -Multiliteracy competence

Table 3. The two groups involved and their activities

The overall responses to the use of ClipFlair were very positive. The UPF team could also detect some issues as on how to implement the tool and on how much “control” and observation by the teaching staff is required so they don’t give up due to technical problems. Below the results of some of the questionnaires, separated by groups:

July 2014 Page | 37


July 2014 Page | 38


July 2014 Page | 39


Below are screenshots of the two activities created on the ClipFlair platform:

July 2014 Page | 40


2.4.3 Imperial College London (ICL) and University College London (UCL) ICL and UCL have carried out the piloting of activities in a variety of contexts: with undergraduate and postgraduate students, with primary and secondary school students, and with independent learners. However, it should be noted that we have encountered many issues to implement the piloting of activities as expected due to the transfer of the Translation Studies Unit from Imperial College London to University College London. As a result of the transfer, the team has not been able to rely on the support of language tutors teaching Arabic, Chinese, Portuguese and Japanese (who stayed at Imperial College London) to pilot activities. This is the main reason why most of the activities piloted by UCL are Spanish activities, as this was the language taught by members of staff who were transferred to UCL. The team attempted to involve tutors from the new institution for the piloting of activities, but this was not possible given the time frame, and given that teaching finished in March. As a result, an attempt has been made to involve independent learners, who have been mainly contacted through social media.

The purpose of this section is to provide detailed information about the piloting activities carried out at UCL and with independent learners. Since the activities carried out at ICL have been included in the final report, these will not be discussed here.

Piloting carried out by UCL

Piloting activities with translation students at UCL

On the 23rd of March 2014, Rocío Baños, from the UCL team, piloted the activity http://studio.clipflair.net/?activity=Friends_en_Rev_C1_ANY.clipflair with 30 students of the MSc in Scientific, Technical and Medical Translation at UCL. The activity was integrated in the module Translating for Voiceover and Dubbing, where students translate a wide range of audiovisual material for the purposes of voiceover and dubbing. In this activity students were asked to translate the clip for dubbing, making sure the translated dialogues sounded natural and spontaneous, and then to revoice the dialogues to find out if their translation fitted in the space provided, and to make sure that it sounded natural. Students translated into a wide range of languages: Chinese, Spanish, Italian, Polish and Portuguese.

The session lasted 2 hours, and the activity was introduced in the last 40 minutes. As there was not enough time for students to complete the task in class, they were asked to finish it at home, to post it on ClipFlair Social Network, and to complete the feedback form. No major issues were experienced during the piloting. There were only minor issues with the revoicing component (some fragments were recorded but could not be heard by students, and there was some overlapping between

July 2014 Page | 41

http://studio.clipflair.net/?activity=Friends_en_Rev_C1_ANY.clipflair


fragments). These issues were reported after the session and looked into by the ClipFlair team in charge of the platform’s development.

A couple of days after piloting the activity, a reminder was sent asking students to complete the activity and to fill in the feedback form (see Appendix 1). Since not many students filled in the feedback form, feedback was sought informally when meeting students in other lectures. Overall, students liked ClipFlair Studio and some of them even tried to complete other activities available in the Gallery. Both during and after the piloting session, some students mentioned that they were not comfortable listening to their own voice. Some mentioned they were not comfortable recording audio in class, but that they felt more at ease doing the recording at home.

Piloting Spanish activities at Lawdale Junior School

On the 23rd of May 2014, Marga Navarrete, from the UCL team, ran four sessions at Lawdale Junior School. Four different sessions were held: two with Y4 students (8-9 years old kids) and two with Y5 students (9-10 years old kids). Each session was attended by about 28-30 pupils.

Two different activities to learn Spanish, designed for primary school children were tested: http://studio.clipflair.net/?activity=Lucas_y_sus_hermanos_Cap-A1A2-SP.clipflair and http://studio.clipflair.net/?activity=Lucas_y_sus_hermanos_Rev-A1A2-SP.clipflair.

The activities consisted of a short clip where an eight year-old boy was interviewed about personal information, family and leisure activities. Pupils really enjoyed the clip (because it was filmed in their local park and also because they could easily relate to the boy in the clip). In the first activity children had to complete the captions, and in the second, they had to dub the main character and change his responses with their own personal information.

The first activity was very successful, most students managed to complete it, and there were no technical problems of any sort. Also, they found ClipFlair Studio easy to use and enjoyed writing the captions. However, when a group of pupils tried to test the second activity based on a revoicing task, problems arose, and none of the students managed to complete the activity successfully. The school borrowed the microphones from another school, and they had not tested them properly, the settings were slightly wrong, so most microphones did not work properly. Although pupils enjoyed the clip, and learnt a lot from it, not being able to complete their task was frustrating.

Almost a hundred students tested these activities. However, there was not enough time to fill in the learner feedback. As it was an online form, some links had to be typed in and there were a lot of questions to be responded to in a very short period of time. For this reason, the teacher assistant and the UCL tutor piloting the activity

July 2014 Page | 42

http://studio.clipflair.net/?activity=Lucas_y_sus_hermanos_Cap-A1A2-SP.clipflair

http://studio.clipflair.net/?activity=Lucas_y_sus_hermanos_Rev-A1A2-SP.clipflair


helped a few students to complete the form, but unfortunately, not as many as we would have liked to.

Use of ClipFlair Social Network

The UCL team has also made use of the social part of the Clipflair platform, mainly through the creation of the group MSc Translation UCL.

Piloting with independent learners

Independent learners were contacted both directly and through social media in order to be able to gather relevant feedback about the platform and about specific activities targeted at independent learners. UCL tutors were also encouraged to ask language learners to test some activities as independent learners. A key aspect raised informally by learners was obtaining support while carrying out the activity. Some learners found the available video tutorials and the manual useful, but others preferred to be able to access this material in their main language, an issue which has been addressed in the project. Another issue raised was obtaining feedback after completing the activity: leaners particularly appreciated being able to check a document with solutions or sample answers. Again, feedback from learners was frequently gathered informally, but some of the responses obtained during the last stages of the project are available in Appendix 2.

July 2014 Page | 43

http://social.clipflair.net/MonoX/Pages/SocialNetworking/Groups/GroupView/mXvhIBnxuUOH-aL0AZIEgg/MSc-Translation-UCL/


2.4.4 Universitatea „Babeș-Bolyai”, Cluj-Napoca, Romania (UBB)

UBB Cluj-Napoca carried out its piloting activity in different contexts: high school students, university students, MA degree students, and professors.

Summary of figures related to the Clipflair Project by UAB



394 questionnaires




Table 2 Summary of UBB team work

Apart from the nine UBB teachers who are members of the Clipflair project and have been in charge of creating and piloting activities, there have been four other high school teachers who have piloted our activities. Two of these teachers belong to the Colegiul Național „Emil Racoviță” (CNER); the others belong to Seminarul Teologic Ortodox (STO) and Liceul Teoretic „Mihai Eminescu” (LTME) Cluj-Napoca. The following table summarizes these figures:

Clipflair members and UBB staff Secondary school teachers

Cristina Varga Diana Todoran (CNER)

July 2014 Page | 44





Clipflair members and UBB staff Secondary school teachers

Cristina Felea Irina Nădășan (LTME)

Adriana Neagu Nicoleta Popa (STO)

Liana Muthu

Sanda Moraru

Alexandra Cotoc

Anamaria Radu

Table 3. Teachers involved in the piloting at UBB

The activities created by the UBB team that have been piloted include the following foreign languages: English, Spanish, Romanian, Catalan, Russian, and Ukrainian. The Russian pilot activities were postponed by Russian Language and Literature Department of UBB by October 2014 and will be carried out as soon as possible. One activity in Norwegian which is not published for the moment in the Clipflair gallery was also carried out because the Nordic Languages Department of UBB would like to involve Clipflair platform in a Norwegian learning project, they will submit by the end of 2014.

July 2014 Page | 45





Figure 1. Use of ClipFlair social network by UBB users

Figure 2. Use of ClipFlair social network by college students in Cluj-Napoca

The UBB team used the social network of the Clipflair platform during the pilot lessons especially for upload the activities of the students. It has resulted in the creation of following groups (the number of members for each group is also mentioned):

• UBB (217);

• Romanian;

July 2014 Page | 46


• Ukrainian; • Russian; • English; • Spanish; • Catalan;

• CNER (62);

• STO (18);

• Liceul Teorectic „Mihai Eminescu” (14);

• NTLD2014 (18);

Therefore, UBB users are organised in 5 main groups and 329 members.

Finally, the UBB team has contacted different authorities:

• Head of Russian Cultural Center in Cluj-Napoca: Iudith Bartalis;

• Head of Liceul Teoretic “Mihai Eminescu”: Dobrescu Simona Raluca

Below are some pictures of the piloting of activities at the UBB. Please find more on the Clipflair Social Network, group UBB.

July 2014 Page | 47


July 2014 Page | 48


2.4.5 University of Deusto

The University of Deusto has collaborated with the Kurutziaga School of Durango to design and test activities. These activities have been developed in Basque. Kurutziaga school is an Associated Partner in ClipFlair project.

Other activities have been piloted at the University of Deusto with students from “Euskal Irakaslegoa”, the Basque language school. Kurutziaga School students who participated in the pilot are in fifth and sixth grade, so they are under 13 years.

Pilot data

The following table shows the list of activities in Basque that have been tested, the number of learners who have tested each activity, the educational institution and the teacher responsible for each piloting activity.

Title of activity (URL link) Number of Learners

Center Teacher

Responsible

NOR NAIZ NI?: http://studio.clipflair.net/?activity=UD_euskera_nornaizni.clipflair

9 Kurutziaga School

Gaizka Uriarte

MARITEILETAKOAREN IPUINA: http://studio.clipflair.net/?activity=UD_euskera_mariteiletako.clipflair

8 Kurutziaga School

Gaizka Uriarte

GABONAK, OLENTZEROAREN KANTAK: http://studio.clipflair.net/?activity=Kurutziaga_euskera_gabonak.clipflai

r

6 Kurutziaga School

Gaizka Uriarte

SURRANDIAK: http://studio.clipflair.ne

t/?activity=Kurutziaga_euskera_Surrandiak.clipflair

18 Kurutziaga School

Gaizka Uriarte

ZEZENAK DIRA: http://studio.clipflair.net/?activity

9 Kurutziaga School

Gaizka Uriarte

July 2014 Page | 49

http://studio.clipflair.net/?activity=UD_euskera_nornaizni.clipflair

http://studio.clipflair.net/?activity=UD_euskera_nornaizni.clipflair

http://studio.clipflair.net/?activity=UD_euskera_mariteiletako.clipflair

http://studio.clipflair.net/?activity=UD_euskera_mariteiletako.clipflair

http://studio.clipflair.net/?activity=Kurutziaga_euskera_gabonak.clipflair



http://studio.clipflair.net/?activity=Kurutziaga_euskera_Surrandiak.clipflair



http://studio.clipflair.net/?activity=Kurutziaga_euskera_zezenakdira.clipflair


=Kurutziaga_euskera_zezenakdira.clipflair

OLENTZEROREN ISTORIOA: http://studio.clipflair.net/?activity=Kurutziaga_Olentzeroren_istorioa.

clipflair

1 Kurutziaga School

Gaizka Uriarte

LAZKAO TXIKI: http://studio.clipflair.net/?activit

y=Kurutziaga_euskera_Lazkao_txiki.clipflair

10 Kurutziaga School

Gaizka Uriarte

ITSUTUTA ABENTURA: http://studio.clipflair.net/?activity=UD_euskera_itsututa.clipflair

2 Kurutziaga School

Gaizka Uriarte

BIZKAIAN TURISMOA EGITEN: http://studio.clipflair.net/?activity=UD_euskera_bizkaia.clipflair

4 University of Deusto

Iratxe Mentxaka

KUTSADURA: http://studio.clipflair.net/?activity=UD_euskera_bizkaia.clipflair

5 University of Deusto

Iratxe Mentxaka

Table 1: List of piloting activities

In conclusion, we have tested 10 ClipFlair activities in Basque and 72 students have participated.

Teachers´ comments

After the piloting, teachers gave us feedback about their experience using ClipFlair.

• It´s difficult to test an activity when students have to record the voice with a microphone, because there are multiple students in the same classroom and it is noisy.

• In their opinion, there should be an option to lock the edited activity, because sometimes students close a window and it´s not possible to recover it.

• They suggest including in the activity the auto-save option, to avoid losing information.

• It would be very useful to use videos from YouTube, without downloading.

July 2014 Page | 50

http://studio.clipflair.net/?activity=Kurutziaga_euskera_zezenakdira.clipflair

http://studio.clipflair.net/?activity=Kurutziaga_Olentzeroren_istorioa.clipflair



http://studio.clipflair.net/?activity=Kurutziaga_euskera_Lazkao_txiki.clipflair



http://studio.clipflair.net/?activity=UD_euskera_itsututa.clipflair

http://studio.clipflair.net/?activity=UD_euskera_itsututa.clipflair

http://studio.clipflair.net/?activity=UD_euskera_bizkaia.clipflair





• It would be very interesting to include external activities, such as Hot potatoes or Google Form, without the need to get out of ClipFlair.

• It would be interesting to have more activity-models and examples. • Finally, teachers pointed out that they consider useful ClipFlair Studio to work areas

such as listening, translation or pronunciation.

July 2014 Page | 51


2.4.6 University of Tallinn

TLU carried out its piloting activity in two different contexts: secondary school students and university students.

Figures related to the Clipflair Project by Tallinn University






The activites were created and piloted by project partners Kristiina Rebane, Kristiina Tedremaa – Levorato, Markus Oder, piloting in the secondary school was carried out by teacher Vittoriano Reno. The piloting took place in Tallinn University. The feedback from the students was mostly positive. The problems occurring were mostly technical (Silverlight installation, Macintosh).

To encourage the Estonian foreign language teachers (FL) to start using audiovisual material in their classes we have been introducing Clipflair project on different occasions:

1. Tallinn Winter School 2013 (07.01 – 25.01.2013).

2. International education day 21.02.2013. A newsstand of different language study options was put up at the university for secondary school students. CLIPFLAIR leaflets were distributed and a mini-lesson was conducted on TV.

July 2014 Page | 52


3. TLU Open doors day 15.03.2013 where the prospective students (secondary school students now) were introduced to the study possibilities at our university. There was an excursion to the language centre, during which Clipflair project was introduced and leaflets distributed.

4. TLU Summer School 2013 (15.07 - 02.08.2013). Target group are foreigners who study Estonian. We targeted about 60 students of Estonian at elementary level.

5. Foreign language teachers methodology days at Tallinn University 22.11.2013 (98 participants).

6. TLU days at our partner schools (students of secondary school, about 1000 students). Autumn 2013.

7. Presenting Clipflair to TLU Institute of Estonian Language and Culture foreign language teachers. 12.12.2013.

8. TLU Winter School 2014 (06.01 – 24.01.2014). Target groups are foreigners who study Estonian. We targeted about 35 students of Estonian at elementary level.

9. Tallinn University Language Centre international conference of foreign languages methodology (about 100 teachers) that was held in 04.04.2014.

July 2014 Page | 53


10. TLU Summer School 2014 (14.07 – 31.07.2014). Target group are foreigners

who study Estonian. We targeted about 40 students of Estonian (elementary and

intermediate levels).

July 2014 Page | 54


2.4.7 University of Warsaw

Pilot study report

The report aims to present the evaluation procedures undertaken by University of Warsaw as part of the ClipFlair project and also to explain the lack of the data from pupils and teachers’ evaluation forms in the project database.

Students’ involvement at the pre-pilot stage

Students of the Institute of Linguistics, University of Warsaw, have been involved in the project work since the beginning of the project. In October2012, a group of 110 students of the Institute of Linguistics, University of Warsaw, were introduced to the idea of Clipflair (CF). 60 of them were involved in the creation of video clips for the project. They worked in groups of four. Due to various levels of technical and content quality, four clips (Święconka, Pyszna zapiekanka, Abecadło Ślubnych obrzędów, Polskie tańce narodowe) were uploaded to the ClipFlair video gallery and used for Cliplair activities.

First piloting stage

At the first piloting stage in 2013 (summer semester 2012/13) the piloting process of ClipFlair activities took place at the Faculty of Polish at the University of Warsaw. 20 students of extramural studies for teachers of Polish as a foreign language used the activities and evaluated them. However, both their academic supervisor and the students refused to fill in CF evaluation forms. They sent evaluation reports to the supervisor and to the Polish partners as written commentaries (see file: Teachers of Polish commenatries March 2013). In their evaluation, the teachers appreciated the attractive and innovative approach to teaching and learning languages, but they also emphasised the technical problems at that stage. They looked at them in a very creative way and suggested other ways the clips could be used for other linguistic and cultural purposes. They stressed that the activities need to be ordered in a traditional way which is clear to the teachers and students: separation of grammar tasks from discussion tasks, clear introductory and revision tasks. They used ClipFlair activities willingly to get to know its idea and implementation, but all explained that as the activities were available on the Internet for free, filling in the forms for the sake of bureaucratic CF procedures were not meaningful to them at all. Their supervisor participated in the evaluation interview and she said that for some teachers of Polish the technology is still a barrier. She asked for authoring tools to

July 2014 Page | 55


make teachers more autonomous in creating activities and adopting them to the students’ needs. As far as the language level was concerned, she opted for activities forA1 and A2 only because most of the teachers of Polish as a foreign language need materials for these levels. For higher levels they may use other materials such as copies of commercial clips. She also complained about the quality of the clip about Warsaw, and some colloquial linguistic forms used in the activity instruction, which are not accepted by Polish purist teachers as they may be misleading to students of Polish as a foreign language.

A teacher at a lower secondary school in Warsaw piloted some activities with 50 students (2 classes). She was interviewed for evaluation, but again neither she nor her students were willing to fill in any evaluation forms. They simply treated the activities as part of the Internet resources, and used them in the way they wanted without an extra effort unrelated to learning and teaching. She tried activities based on Rosetta Stone’s clip from the Khan Academy as it related to the topics they worked on History class. Although the students are familiar with the topic, the language level was too high for them. However, she evaluated ClipFlair activities as an interesting approach to learning and teaching.

50 third-year students of the Institute Applied Linguistics Bachelor Programme participating in a course Methodology of teaching English (pre-service teachers of the following languages (French, German, Spanish and Russian)) were shown how to use audiovisual materials in ClipFlair activities to develop all four language skills and audiovisual skills. As the group was big, 27 students, and the computer lab is only for 20, they could use the activities at home computers, because ClipFlair was not available at their mobile devices in class. As the classes did not take place in the computer laboratory, group work was only possible. In class they were discussing activities based on Maggie the Witch and Rosetta Stone clips. The evaluation was done in the form of focus group interviews. The students presented balanced opinions. On the one hand, they appreciated the interactivity of the tasks, and the clear and natural context for language learning, e.g. a visit in the museum in the Rosetta Stone video. On the other hand, they noticed the gap between the time pupils get to know about the Rosetta Stone at school and their language level not high enough to understand the text with ease.

The second piloting stage

Winter semester 2013/14.

July 2014 Page | 56


The students of the Institute of Applied Linguistics were involved at the piloting stage at different levels and through different courses.

110 participated in a course Intermedial Teaching of Languages at the University of Warsaw Masters Programme. They were introduced to the ClipFlair project aims and activities. As the course was based on students’ access to their own mobile devices (Bring your own device approach), some of them couldn’t get access to ClipFlair activities through Android or iOS system on tablets in class. They all joined the ClipFlair community at social group Students at Warsaw University and their evaluation of various ClipFlair activities prepared in various languages is available there. The course puts emphasis on reflective and autonomous approach to learning and teaching. Thus, forms of documenting work were negotiated with the students. Oout of thetwo forms of evaluating the activities social media or forms, they chose social media as more qualitative. One of them reported later in an informal talk with the teacher that she used ClipFlair activities with some of her private learners, and they found them attractive, enjoyable and effective.

50 third-year students of the Institute Applied Linguistics Bachelor Programme participating in the course Evaluation in glottodidactics (pre-service teachers of English) evaluated various elements of ClipFlair activities separately and activities as such. The ClipFlair activities were introduced to illustrate evaluation of audiovisual materials, listening and writing activities as well as audiovisual writing and speaking. The criteria for evaluating clips worked out within ClipFlair methodology were introduced as general tools for clip evaluation. They evaluated the clips as audiovisual materials to be used in language teaching and learning. They were all given the ClipFlair evaluation form address but they did not use it. 10 of the students provided written report as a document for their extra voluntary work. They are available in the zip file: Students’ evaluation October November 2013. They also evaluated ClipFlair activities, in group discussions in class, from the perspective of developing different skills as part of the use of various class tasks, but they did not prepare any written documents to summarise the results of the discussions. They also evaluated the authoring functionalities of the portal from the perspective of teachers who wanted to create their own activities. The appreciated the idea of using either films prepared by students or with students from the workstation without the need to publish them on the Internet.

Summer semester 2013/14

July 2014 Page | 57


Course in Current tendencies in glottodidactics – lecture

130 students participated in the course. ClipFlair was used to illustrate current trends in methodology of teaching languages. However, because of the type of the course, that is the lecture, the students did not use ClipFlair activities in class. They evaluated the activities in the interactive way during the lecture. At this stage of their professional development, they could evaluate the materials both from the students’ perspective and from the teacher’s perspective. And it turned out the perspectives differ. As students they were positive about the idea of using audiovisual materials by adding captioning and revoicing. They stressed the role of interlingual translation in learning with audiovisual materials. However, as teachers they would recommend the materials to pupils for individual work at home, but in class they will use them only on the condition they had easy access to computers, which is unlikely, and if the pupils were computer literate enough to focus on the language task. As the only course requirement for them was to pass the final exam at the end of the semester, it was impossible to ask them to formally evaluate the activities through filling in the ClipFlair forms.

The table below presents the number of users involved in the creation and evaluation of ClipFlair activities

Stage Time Number of secondary school students

Number of in-service teachers

Number of pre-service teachers

Production Winter sem.2012/2013

0 0 60

First piloting Summer sem. 2012/13

50 20 150

Second piloting stage Winter sem. 2013/2014

0 0 160

Second piloting stage Summer 0 0 130

July 2014 Page | 58


sem.2013/2014

Total 50 20 500

Final explanations

To explain the context and reasons why students and teachers refused to fill in the evaluation forms, it is worth mentioning that in those days they were paid for participation in another project, and they also regularly received money for participation in surveys introduced by marketing companies. Thus, they were willingly learning the methodology how to use audiovisual materials in language learning, how to use interactive learning environment for learning and teaching languages, how to include multimodal translation into teaching and learning with technology, but they were not willing to invest their time and effort in filling in project questionnaires for free. As the activities were available on the Internet, the pupils used them in a way they wanted.

At the institutional level it was also discussed to include the use of a ClipFlair activity as an obligatory task for one lesson taught by pre-service teachers during their pedagogical internship, which takes place in either primary or secondary schools. The university internship supervisors, after consultations with mentors at schools, refused to include this task, because of technical problems. The mentors were not able to provide the students with access to school computer laboratory. As the requirements for all students have to be the same, the task was not implemented.

A teacher training centre in Warsaw was asked to include the ClipFlair activities in their teacher training courses. They refused to do it as they had to follow a strict plan and syllabi of the courses which had been approved the management team and the supervising local authorities a year before.

As the reflective and autonomous approach to teacher training is implemented in some of the courses presented above, the students have a selection of tasks to be done as course requirements, or optional forms of documenting the work done with the emphasis on how meaningful the task is for them. They selected a narrative form of evaluation in a social group as more meaningful than filling project forms.

Conclusions

July 2014 Page | 59


The project activities were introduced to 20 teachers of Polish as a foreign language, 50 pupils and 500 students of the Institute of Applied Linguistics and evaluated by them with the use of various qualitative tools, e.g. ClipFlair social group, interviews, focus group interviews, class discussions. The evaluation is more qualitative than quantitative in its nature, but well fitted to the course requirements and conditions. From this perspective, qualitative evaluation is complementary to quantitative one done by other partners. This also shows that ClipFlair activities may constitute a part of regular methodology courses for pre-service teachers of languages without changing syllabi of such courses. Piloting ClipFlair activities with pre-service teachers of various languages adds an extra value to the implementation of ClipFlair project results, which can also be perceived as extra unpredicted result of the ClipFlair project.

July 2014 Page | 60


2.4.8 National University of Ireland, Galway (NUI Galway)

NUI Galway carried out its piloting activity in a variety of contexts: with undergraduate university students, independent learners, adult learners in e-learning environments and secondary school students.

Summary of figures related to the ClipFlair Project by NUI Galway

28 activities created + 15 extra activities created for Italian, 1 French, 1

German*


Students involved: approx. 400 students (some of these students tested

several activities)

questionnaires answered: 54 **

8 teachers and students involved in the creation of activities

5 teachers involved in the piloting with students

3 extra teachers were involved in the peer-review process and also individually piloted Irish activities (Lisa Ní Fhlatharta, Belinda McHale & Fiona Ní Chualáin)

Table 1. Summary of NUI Galway team work

* Additional activities in French and German were created specifically for piloting purposes with Mercy College Secondary School.

**Because some activities were tested in tandem with Associate Partners (specifically Università degli Studi di Pavia and Uned, many answered questionnaires do not include reference to NUI Galway or to one of our teachers, for that reason they are not included here).

July 2014 Page | 61


In addition to the three NUI Galway teachers who are members of the ClipFlair project, one additional teacher (Dr Jennifer Lertola) created and piloted activities with both teacher-dependent and independent learners. In particular, during the Academic Year 2013-2014, Dr Lertola carried out a two-semester pilot of activities with online mature students in order to test the feasibility of use of ClipFlair in e-learning environments. This was successful and it will be repeated again during the Academic Year 2014-2015. 13 students participated in this pilot, which tested 9 different activities, for a total of 117 tests.

Two more teachers (Sara Buscio and Alessandro Luchetti) piloted activities within NUI Galway, while three others peer-reviewed and carried out individual pilots of some of the Irish-language activities. A student (Sinéad Nic Aodha) and two other members of staff (Éamon Ó Cofaigh, Pilar Alderete-Díez) also helped in the creation of a couple of the activities. Outside NUI Galway, our activities were piloted at the following institutions:

- Dublin Institute of Technology - Mercy College Secondary School, Sligo - Kazimierz Wielki University, Bydgoszcz, Poland - Università degli Studi di Pavia, Italy (in collaboration with Associate Partner), - Europass, Florence, Italy - Uned, Madrid (in collaboration with Associate Partner)

Clipflair members NUI Galway teachers Outside NUI Galway

Laura McLoughlin Jennifer Lertola Dublin Institute of Technology

Dorothy Ní Uigín Sara Buscio Mercy College, Sligo (secondary school)

Rose Ní Dhubhda

Alessandro Luchetti

Kazimierz Wielki University, Bydgoszcz,

Poland

Università degli Studi di Pavia, Italy

Europass, Florence, Italy

July 2014 Page | 62


Clipflair members NUI Galway teachers Outside NUI Galway

Uned, Madrid, Spain

Table 2. Teachers and centres involved in the piloting by NUI Galway

Unfortunately not all students completed a feedback questionnaire or registered in the Social. In the case of the Mercy College Secondary School, teachers advised us that they were not in favour of the registration of under-age students. In other cases, for example the online course, the same students were involved in the pilot of several activities during two semesters. These students registered and filled in questionnaires only once.

The activities piloted by the NUI Galway team (also in close collaboration with Associate Partners) include a number of languages to reflect the interests of the students involved in the pilot. They include the following languages and language combinations:

ANY LANGUAGE - 1

ENGLISH – intralingual - 13

IRISH – intralingual - 3

ITALIAN – intralingual 15

ENGLISH-ITALIAN - 10

ENGLISH-FRENCH 1

ENGLISH-GERMAN 1

SPANISH - 1

Use of ClipFlair social network

The NUI Galway team has also made use of the social part of the Clipflair platform. It has resulted in the creation of the following groups:

• NUIG 17

July 2014 Page | 63


• Learning English 9

• Learning Irish 3

• Italian is fun 22

• Bydgoszcz 6

• UNED 65 (In collaboration with Associate Partner)

• Commercials to Learn English 2 (In collaboration with Associate Partner)

• RECORDS Project 9 (In collaboration with Associate Partner)

While piloting activities in the Mercy College Secondary School and presenting ClipFlair to the language teachers, it was discovered that the www.clipflair.net website was not accessible from within the school. The teachers explained that, for security reasons, only approved websites can be accessed from within schools and provided contact details so that the ClipFlair website could be proposed for addition to the list. The PDST (Technology in Education Section) was therefore contacted, www.clipflair.net was vetted, approved and added to the list and it is now accessible from all secondary schools in Ireland.

Finally, the NUI Galway team has contacted a number of different players in the field of Education, who have multiplying capacity – the School of Education within our own university for example; a national organisation – the Professional Development Service for Teachers (PDST) as well as some individual players (teachers, schools etc). Some have become Associate Partners while the project was being developed:

• One Voice for Languages (Associate Partner)

• Association of Teachers of Italian (Associate Partner)

In addition, the following bodies were Associate Partners from the beginning of the project:

• Post-Primary Initiative for Languages

• An Chomhairle um Oideachas Gaeltachta & Gaelscolaíochta (COGG)

July 2014 Page | 64

http://www.clipflair.net/

http://www.clipflair.net/


3 COMPARISON BETWEEN PILOT PHASES

To conclude we will briefly compare the data obtained after the two pilot phases. It

has to be noted that not all users at the pilot phase completed the questionnaire,

which means that the below results are partial. The complete results are presented in section 2.4 Pilot use by partner.

3.1 Participating Institutions

The number of participating centres in the second pilot phase were twice as many in

relation to those participating in the first pilot phase: from 7 to 16. In the first pilot

phase all centres were universities, mostly ClipFlair partners. In the second phase,

besides 10 university partners, 3 more universities participated, plus 2 primary

schools and 1 high school.

3.2 Participants

The number of teachers piloting activities also doubled: from 18 to 37. The number of

questionnaires answered by students, i.e., completed activities, increased from 318

to 1173.

3.3 Activities

The number of activities piloted in the second phase increased tremendously.

The quantity of piloted activities in the second phase (84) is four times larger than

that of the first phase (20). The range of languages worked on in the activities almost

doubled, going from 7 to 12. It is worth highlighting that only 20.7% of activities were

designed to practise English, 13.8% were designed to practise non-European

languages (Chinese, Japanese and Arabic), and 13% were designed to practise

translation (different language combinations).

July 2014 Page | 65


3.4 Teachers' results

The data obtained from the first and second pilot phases are very similar as far as

teachers' results are concerned. The results for the questionsWhere did students do

the activity? and How often teachers use audiovisual materials? are almost identical

to those in the interim report. RegardingTechnical problems, a decrease of 3% has

been observed.

3.5 Students' results

The most relevant data observed when comparing the first and second pilot phases

is the big increase in the number of activities completed (as observed in the

increased number of questionnaires filled in by students, going from 318 to 1173).

Another relevant piece of data is the students' country of origin and age. In relation to

origin, students participating in the first pilot phase came from 17 different countries,

while there were 30 different countries in the second phase. Regarding students' age,

the fact that in the second phase primary and secondary students took part in the

piloting, has made this data change, with the range of 18-35 years of age

representing 93% in the first phase and 74% in the second.

July 2014 Page | 66


APPENDIX 1. Teachers’ questionnaire

July 2014 Page | 67


July 2014 Page | 68


July 2014 Page | 69


APPENDIX 2. Students’ questionnaire

July 2014 Page | 70


July 2014 Page | 71


July 2014 Page | 72


July 2014 Page | 73


APPENDIX 3. Pictures from piloted activities

Some teachers took pictures during the piloting of activities in the language labs. Here are some examples.

July 2014 Page | 74


APPENDIX 4: Reminder sent to students via email

July 2014 Page | 75

1

Banos Pinero, Rocio

From: Banos Pinero, RocioSent: 26 March 2014 10:15To: Felices Gutierrez, Ana; Barcena Panero, Ana; Zamagni, Benedetta; Gioia, Cecilia;

Song, Chang; Di Stadio, Elisabetta; Zhang, Feifei; Zhou, Heran; Chen, I-Chun; Fang, Jing; Acha Villaro, Judit; Filicicchia, Laura; Kozinski, Marcin; Napoli, Marianna; Cecino, Martina; Zhang, Qian; Banos Pinero, Rocio; Chen, Ran; Piras, Simona; Haro Guerra, Sofia; Poerschke Freitas, Thaise; Ning, Tianbai; Gaudenzi, Valeria; Labianca, Viviana; Cai, Wangyu; Zhou, Wei; Tan, Tina; Jin, Xinchun; Ni, Sharon; Peng, Yao; Zhao, Yingwen; Chou, Yu-Hua; Liang, Zhenming

Cc: Navarrete Ramirez-Montesinos, MargaSubject: IMPORTANT: TRANG012 Task to be completed

Dear all,

This is a reminder about the ClipFlair task to be completed for TRANG012. I know it’s a long email, but please read it to the end.

What do you have to do?

1) Translate Friends dialogue into your target language, making sure that your translation contains features typical of spontaneous conversation. For this you will have to follow these steps:

a. Open the activity by going to http://studio.clipflair.net/?activity=Friends_en_Rev_C1_ANY.clipflair b. If you want to carry on where you left it on Monday, load your work there by clicking on the button

Load Activity from file, at the bottom of the screen

. Select the file you saved after the session on Monday. It will have a .clipflair extension.

c. If you prefer to, you can start from scratch. In this case, make sure you save your work frequently by clicking on the floppy disk icon. When resaving your work, it will ask you to replace the file you created the first time you saved it, which is fine.

d. Carry on with your work: enter the translation in the caption spaces provided (the timing has been set up for you, but you can change this if you want to). Remember that this text is not for subtitling in this case, but to help you record the dialogue.

e. Once you have entered your translation, revoice each of the captions separately by clicking on the

recording button ( ). When you have finished reading your translation aloud, click on . f. You don’t have to play the video as you record! g. Once you have recorded a few utterances, click on Play to see how your translation will sound. Bear

in mind that synchronisation won’t be perfect, but this is not important here (you’re not a voice talent!). What matters is that you realise: 1) if your translation sounds oral; 2) if your translation fits within the time given.

2) IMPORTANT: Fill in the feedback form available at: http://tinyurl.com/clipflairfeedback. This is very important as it will enable us to improve the platform to be able to use it for teaching purposes. By filling in this form you will be contributing to the ClipFlair project by helping to develop this free online and easy to use platform for subtitling and dubbing! ***When filling in this form, you will be asked to include the name of the tutor (Rocío Banos), as well as the URL of the activity: http://studio.clipflair.net/?activity=Friends_en_Rev_C1_ANY.clipflair

3) OPTIONAL: Upload your activity in ClipFlair Social Network a. Go to http://social.clipflair.net/. b. If you haven't registered yet, do so by clicking on

http://social.clipflair.net/login.aspx?ReturnUrl=%2f. c. Go to the MSc Translation group and post your activity there.

2

Why should I post my activity there? I’ll try my best to make sure that feedback is provided to those who post their activities on the platform. Also, you will be able to check what other peers have done and learn from others’ work.

If you have any questions, do not hesitate to contact me!

I hope you enjoy working with ClipFlair.

Rocío

Dr Rocío Baños Lecturer in Translation Centre for Translation Studies (CenTraS) University College London Room 206, 50 Gordon Square London WC1H 0PQ Tel. +44 (0)2076799361 [email protected] http://www.ucl.ac.uk/centras