From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese context

From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese contextJoão Caramelo, Susana Coimbra, Marta [email protected], [email protected], [email protected] LIBE Final Event

December 2015PROJECT REF. NO. 543058-LLP-1-2013-1-IT-KA3-KA3MPThis project has been funded with support from the European Commission. This communication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

Index

Introduction Key competences and skills UPORTO Courses Pilot study: global quantitative results Pilot study: global qualitative results Final reflection

December 2015

Introduction

European project LIBE: common framework

December 2015

Problem tackled:European countries dealing with issues deriving from low educational achievers(16-24) in literacy, numeracy and problem solving (OECD, 2013).

Introduction

December 2015

• Cognitive skills and reflective thought processes.• Using tools interactively.• Interacting in

heterogeneous groups.(Rychen & Salganik, 2003; Oates, 2003;

OECD, 2013)

• Basic ICT skills.• Literacy.• Numeracy.• Managing information in

digital environments.

Linked to• Problem solving in

technology rich environment.

(PIAAC, 2013; OECD, 2013)

Which are the competences and skills that LIBE courses looks to develop?Competences and skills needed for people aged 16-24, with a profile of low achievers and at risk of social exclusion as a result of not having completed basic education levels and not having entered the labor market:

Introduction

December 2015

LIBE PROPOSES TO CREATE AND IMPLEMENT E-LEARNING COURSES AIMED TO HELP YOUTH AND YOUNG ADULTS (16-24) WHO STRUGGLE TO MASTER THESE BASIC SKILLS.LIBE Courses foresee…Cognitive strategies and contents for literacy, numeracy, and problem solving skills in technology rich environments (OECD; PIAAC).

“Computer and information literacy” that integrates e-skills - ICT practitioner skills; ICT user skills; and e-Business skills (European Commission, 2013).

Key Competences and Skills: focus groups

Focus groups on key Competences and Skills for young low achievers’: hearing the voices of Portuguese students, trainers and teachers.

December 2015

12 teachers/trainers:

2 focus groups

7 students “low achievers”:

1 focus group

Key Competences and Skills

Key Competences and Skills most relevant for young low achievers:

December 2015

1.Litera

cy

2. ICT competen

ces+ 3. Social competen

ces+ 4. Pedagogi

cal Support

+5.

Self-efficay

+

• Topic: Awareness about food.

• Skills and Competences focus: Problem solving in technological rich environments + Literacy.

• Topic: reduce waste through 3Rs initiatives – Recicle, Reuse, Reduce.

• Skills and Competences focus: Literacy.

UPORTO Courses

December 2015

UPORTO 2 LIBE courses

UPORTO Courses

December 2015

Pre-test4 activities

Course11 activities2 Videos

Post-test4 activities

Pre-test6 activities

Course11 activities2 Videos

Post-test6 activities

Piloting UPORTO Courses

December 2015

Piloting of UPORTO LIBE courses in Portugal

246 students total175 finished both courses: 91 male | 84 female Age: 14 to 23

M=17.41 | SD=1,43

3 Schools

Pilot study: Dimensions, examples of questions and descriptive statistics

INITIAL QUESTIONNAIRE Questions | Scales of answers

Self-efficacy toward ICT

How well can you do each of the following tasks on a computer?•Move files from one place to another in a computer?•Download files of software from the internet?(1) I can do this very well by myself – (4) I don’t know what this means

Attitudes toward potential of internet

Think about your experience with computers: to what extent do you agree with the following statements?•Using the computer for learning is troublesome.•The internet is a great resource where to obtain information from, for my school work.(1) Strongly agree – (4) Strongly disagree

Attitudes toward limits of internet

December 2015

RESULTS N Minimum Maximum Mean Std.

DeviationSelf-efficacy toward ICT 175 2,75 4,00 3,70 ,26Attitudes toward potential of internet

175 2,33 4,00 3,67 ,40Attitudes toward limits of internet

175 1,00 4,00 2,34 ,63


FINAL QUESTIONNAIRE Questions | Scales of answers

Difficulty of questions Course 1

What did you think about the difficulty of the questions in this course? (1) All Easy – (4) All Challenging

Difficulty of questions Course 2Motivation Course 1 Did you feel motivated while working on this

course?(1) Strongly agree – (4) Strongly disagreeMotivation Course 2

December 2015


DeviationDifficulty of questions Course 1

175

1,00 4,00 1,74 ,84Difficulty of questions Course 2

175

1,00 4,00 1,63 ,77

Motivation Course 1 175

1,00 4,00 3,21 ,69

Motivation Course 2 175

1,00 4,00 3,17 ,75


FINAL QUESTIONNAIRE Questions | Scales of answers

Usability Course 1 What do you think about the e-learning system of this course?(1) Strongly disagree – (5) Strongly agreeUsability Course 2

Final assessment Course 1 Overall, you would rate the user-friendliness of this course as:(1) Worst Imaginable – (7) Best Imaginable Final assessment Course 2

December 2015


DeviationUsability Course 1 17

51,00 5,00 3,63 ,79

Usability Course 2 175

1,00 5,00 3,28 ,73

Final assessment Course 1 175

1,00 7,00 5,34 1,24

Final assessment Course 2 175

1,00 7,00 5,18 1,34

Pilot study: Predictors of assessment Course 1

December 2015(F=13.762; p≤.001; Adjusted R Square=.382)

Model

Unstandardized

Coefficients

Standar

Coeffic. t Sig.

Collinearity

Statistics

B Std. Error Beta Tol VIF

,99 1,65 ,60 ,55 Difficulty Course 1 ,04 ,11 ,02 ,35 ,73 ,79 1,27

Motivation Course 1 1,01 ,13 ,57 8,08 ,000 ,76 1,31

Usability Course 1 ,30 ,11 ,18 2,65 ,009 ,79 1,27

ICT Self-efficacy ,15 ,32 ,03 ,47 ,64 ,84 1,19


-,10 ,20 -,03 -,49 ,62 ,91 1,10


,08 ,13 ,04 ,59 ,56 ,82 1,22

Age -,03 ,06 -,03 -,45 ,65 ,95 1,06Gender_Male ,00 ,16 ,00 ,02 ,98 ,91 1,10

Pilot study: Predictors of assessment Course 2

December 2015(F=4.303; p≤.001; Adjusted R Square=.138)

Model

Unstandardized

Coefficients

StandarCoeffic. t Sig.

Collinearity

StatisticsB Std.

Error Beta Tol VIF

1

1,45 2,11 ,69 ,49

Difficulty Course 2 -,22 ,13 -,14 -1,68

,10 ,76 1,31

Motivation Course 2 ,52 ,15 ,30 3,54 ,001 ,73 1,37

Usability Course 2 ,09 ,14 ,05 ,62 ,54 ,90 1,11

ICT Self-efficacy ,54 ,40 ,11 1,35 ,18 ,86 1,17


-,07 ,25 -,02 -,26 ,80 ,92 1,09


,19 ,16 ,09 1,15 ,25 ,88 1,14

Age ,00 ,07 ,00 ,04 ,97 ,93 1,08

Gender: Male -,16 ,20 -,06 -,79 ,43 ,90 1,12

Pilot study global results: Positive feedback from students to course 1 and 2

December 2015

System related User experience Course related

E-learning system.

Learning experience.

Motivation.Interest.

Relevant topics.More visual content.Less number os

questions.Less amount of plain

text.

“Allows you to learn important things in a more playful way.”

“It was a fun and educational experience”

“Interesting, appealing and important for day to day.”

"This course is important because there are things that we should all know and pass on, from generation to generation, because it’s something that contributes positively to all of us and to protect the planet.“

. . .

Pilot study global results: Negative feedback from students to course 1 and 2

December 2015

System related User experience Course related

Technical errors. Usability. Navigation

buttons: check/next/go back.

Demotivation. Takes too long to

finish the course.

Less visual content. Too many activities. Greater amounts of

plain text. Repetition of pre-test

and post-test.

“Has errors. Needs more interactive questions and more information.”

"It has to be taken into account that the platforms was not functioning properly…”

“Too extense, boring and confusing”

“Make the website graphics more captivating, to make the user feel even more at ease, and not feel it’s such a dull thing.”

“Has too much text. Some questions are confusing. Should have more images.”

“Don’t repeat the tests.”

. .

.

Final reflection

December 2015

Receptivity to learning experience through e-learning.

Mismatch between the sense of self-efficacy regarding ICT and its usefulness in the immediate academic achievement and lifelong learning.

Importance of social and peer support as a form of pedagogical support in e-learning, and an intuitive choice for collaborative work, developing social competences.

Topics are more relevant when creating a meaningful relation between familiarity and usefulness.

Heterogeneity of “low achievers”.

Thank you!

LIBE Final Event December 2015

PROJECT REF. NO. 543058-LLP-1-2013-1-IT-KA3-KA3MPThis project has been funded with support from the European Commission. This communication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

João Caramelo, Susana Coimbra, Marta [email protected], [email protected], [email protected]

Education

From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese context