Assessing digital literacy levels: the design of an instrument

Assessing digital literacy levels: the design of an instrument

AILA 2014Brisbane – Australia -

AugustRosinda de Castro

Guerra RamosF. Katherine Asega

Simone T. M. Ramos

This research aims to investigate how to measure the level of Digital Literacy (DL) of

in-service English teachers in Brazil in order to promote their digital inclusion.

RESEARCH OBJECTIVE

MOTIVATION

SOCIAL:Changes in society.

Transition between "old and new, real and virtual,

the printed and digital" (Buzato, 2006) requires a

set of competences related to Technologies.

Educating efficient digital citizens.

MOTIVATION

Three reasons to include ICT in education:

1. Potential benefits of ICT for teaching and learning.

2. The need to acquire Digital Competence to be functional in our knowledge society (Eshet-Alkalai, 2004).

3. Digital inclusion depends more on knowledge and skills than on access and use (Erstad, 2010b).

EDUCATIONAL:Integration of ICT in practice.


MOTIVATION

PROFESSIONAL:The need for teacher development.


• To become digitally literate.• To help students to become collaborative, problem solving, creative learners through using ICT.

BUT...What level of digital literacy

should teachers have to make a more appropriate

use of ICTs in practice?

Transition between "old and new, real and virtual, the printed and digital" (Buzato, 2006) requires a set of

competences related to Technologies.

Three reasons to include ICT in education:1. Potential benefits of ICT for teaching and learning.2. The need to acquire Digital Competence to be functional

in our knowledge society (Eshet-Alkalai, 2004).3. Digital inclusion depends more on knowledge and skills

than on access and use (Erstad, 2010b).

Challenges: transition between "old and new, real and virtual, the printed and digital" (Buzato, 2006).

Moreover, ICTs enable social inclusion, and its use should not be considered just as a tool, but as social practice that may

mediate the process of English teaching and learning. In this new scenario distance education courses have emerged

together with the necessity of promoting Digital Literacy (DL). If on one hand the demand for such courses has

increased significantly, on the other hand the design of online materials, especially for language teaching has been lagging behind. There is still great concern about what level

of DL teachers should have so as to make a more appropriate use of ICTs regarding online teaching and

digital teaching materials.

SOCIAL REASONS

Our objective is to present the process of designing

an assessment instrument to evaluate

the level of DL of Brazilian English Teachers in-service.

PRESENTATION OBJECTIVE

Digital Literacy and its concepts.Ala-Mutka, K. (2011); Bawden, D. (2001); Bawden, D. (2008); Erstad, O. (2010b); Eshet-Alkalai, Y. (2004); Eshet-Alkalai, Y., & Chajut, E. (2010); European Commission (2013); European Commission (2012); Martin, A. (2006); Martin, A., & Grudziecki, J. (2006); UNESCO (2011).

THEORETICAL FOUNDATION

Digital Competence.Ala-Mutka, K. (2011); Bawden, D. (2001); (Eshet-Alkalai, 2004). European Commission (2013); European Commission (2012).

Assessment.Bachman & Palmer (1996).

“Digital Literacy is the awareness, attitude and ability of individuals to appropriately use digital tools and

facilities to identify, access, manage, integrate, evaluate, analyze and synthesize digital resources,

construct new knowledge, create media expressions, and communicate with others, in the context of specific

life situations, in order to enable constructive social action; and to reflect upon this process.”

(Martin & Grudziecki, 2006).


Digital Literacy Concept

"Digital Competence involves the confident and critical use of Information Society Technology (IST) for work, leisure and communication. It is underpinned by basic skills in ICT: the use of computers to retrieve, assess,

store, produce, present and exchange information, and to communicate and participate in collaborative

networks via the Internet." (European Parliament and the Council, 2006).


Digital Competence Concept

•The DIGCOMP study was launched by European Commission's Joint Research Centre (JRC) - Prospective Technological Studies (IPTS).

•Carried out between January 2011 and December 2012.

•AIM: to identify descriptors and develop conceptual framework of Digital Competence. • RESULTS: two different interrelated outputs - a self-

assessment grid and a framework.


Digital Competence Project (DIGCOMP)

Self-Assessment Grid• Proposes the areas of Digital

Competence and descriptors for three proficiency levels.

• It can be used as a tool to:– identify the level of digital

competence of each citizen;

– to understand how to improve digital competence.


Framework• Identifies (for each area) the related competences.• Provides descriptors on three levels and examples knowledge, attitudes and skills. • It can be used by curricula and developers to develop the digital competence of a specific target group.

Digital Competence Project (DIGCOMP)


Digital Competence – Areas and Competence


Digital Competence Descriptors

DIGITAL LITERACY TEST DESIGN

Test Design

Putting into Practice - Stage 3

Conceptual MappingBrowsing

Assessment

Conceptual MappingBrowsing

Digital Literacy

Available Tests

Checking and comparing

Conceptual Mapping

Looking for Descriptors

Methodology

Online Proficiency Test that will measure the Digital Literacy (DL) considering three different levels of knowledge, attitudes and skills: basic, intermediate, advanced.

Digital Literacy Test description

Aims to identify the level of DL of in-service English teachers in Brazil and inform the participants how to improve digital competence according to the stated needs.


Simone Ramos

não sei se podemos dizer o que está escrito em vermelho pq fica com cara de diagnóstico.pergunto: é proificiency ou diagnóstico o que queremos???

PHASE 1 Computer Literacy

TOOL FOCUS TEST

•Aims to evaluate: - the skills to operate a variety of

computer applications packages (word processing, databases, spreadsheets); - general IT skills, such as copying disks and generating hard-copy printout. first step towards advanced

knowledge, skills and attitudes.

PHASE 2 Digital Competence

KNOWLEDGE, ATTITUDE AND SKILL FOCUS TEST

•Aims to evaluate digital competence into three levels: basic, intermediate, advanced.

Taking into account 5 different areas: information,

communication, content-creation, safety and problem-solving.


General Description

Question Description

PHASE 1 – Computer Literacy•Total of questions: 15 - 5 questions of Computer Basics

(computer devices and its functions). - 10 questions of Computer Concept (software and programs usage). - Total amount of time: 7 min.

Close ended questions, choose from a list of choice to identify the

computer skills.

Question Sample

1. Which computer devices(components) do you know how to use?Choose the skills that best describe your experience:

( ) Monitor( ) Touch Screen( ) Keyboard( ) Mouse( ) Touchpad( ) CD-Rom drive( ) USB drive( ) External drive( ) Printer( ) Scanner


Question Description

PHASE 2 – Digital Competence•Total of questions: 22 - 3 questions of Information: browsing, evaluating and storing. - 6 questions of Communication: interaction, sharing, engaging, collaborating, online norms and digital identity. - 5 questions of Content-creation: developing content, integrating resources, copyright, program modification. - 4 questions of Safety: protecting devices, personal data, health, environment. - 4 questions of Problem solving: solving technical problems, Identifying needs and technological responses, innovating and creatively using technology. •Total amount of time: 22 min.

Close ended questions, multiple choice with questions that

represent the competence level.


Question Sample

PHASE 2 – Digital CompetenceSafety area


Basic levelIntermediate levelAdvanced level Close

Need to start from Computer Literacy

Each and every area will have its own

result, thought they are related.

Preliminary Results - Stage 3: pilot phase PHASE 1 – Computer Literacy

•Submitted the questions to analysis and interviewed the participants: - 2 different Information Technology specialists. - 2 different English teachers (basic level of experience).•The results: - one specialist considered the questions obsolete due to used terminology use and technology advances. The other suggested some terms adjustment. - both English Teachers know how to use the computer but they are not familiar with the used terms.•Conclusion: we are rewriting the questions based on the observation, adjusting the terms and focusing more on the use of software and programs based on daily task and use experience.


PHASE 2 – Digital Competence•Will Submit the questions to: - 2 different Information Technology specialists. - 10 different English teachers (from basic to advanced level).

•The expected results: - to map the level proficiency. - to adjust questions and answers. - to map exam time consuming. - to experience a different online interface based on programming and automatic results and reports.


Preliminary Results - Stage 3: pilot phase

FRAMEWORK • The framework is based on the recognition that both cognitive andtechnical skills are necessary for people to be functional in a digital

society.• Target group: adults.• It supports a vision of digital• literacy as the ability to create, communicate and collaborate to

organise and produce• information; to understand and apply knowledge of the functions of

ICT; to use ICT for• thinking and learning; and to develop a critical appreciation of the

role of ICT in society.• Target group: primary and lower secondary schools.

Simone Ramos

não sei o que fazer com este slide.

Ala-Mutka, K. (2011). Mapping Digital Competence: Towards a Conceptual Understanding. Seville: JRC-IPTS. Retrieved from http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=4699

Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.

Bawden, D. (2001). Information and digital literacies: a review of concepts. Journal of Documentation, 57(2), 218-259.

Bawden, D. (2008). Origins and Concepts Of Digital Literacy. In C. Lankshear & M. Knobel (Eds.), Digital Literacies: Concepts, Policies & Practices (pp. 17-32).

Erstad, O. (2010a). Conceptions of Technology Literacy and Fluency. In P. Penelope, B. Eva & M. Barry (Eds.), International Encyclopedia of Education (pp. 34-41). Oxford: Elsevier.

Eshet-Alkalai, Y. (2004). Digital Literacy. A Conceptual Framework for Survival Skills in the Digital Era. Journal of Educational Multimedia & Hypermedia, 13(1), 93-106.

Eshet-Alkalai, Y., & Chajut, E. (2010). You can teach old dogs new tricks: The factors that affect changes over time in digital literacy. Journal of Information Technology Education, 9, 173-181.

REFERENCES

http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=4699



European Commission (2011). Mapping Digital Competence: Towards a Conceptual Understanding. Luxembourg: Publications Office of the European Union.

European Commission (2011). Mapping Digital Competence: Towards a Conceptual Understanding. Luxembourg: Publications Office of the European Union.

European Commission (2012). Digital Competence in Practice: An Analysis of Frameworks. Luxembourg: Publications Office of the European Union.

European Commission (2013). DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe.

Martin, A. (2006). Literacies for the Digital Age. In A. Martin & D. Madigan (Eds.), Digital Literacies for Learning (pp. 3-25). London: Facet.

Martin, A., & Grudziecki, J. (2006). DigEuLit: Concepts and Tools for Digital Literacy Development. ITALICS: Innovations in Teaching & Learning in Information & Computer Sciences, 5(4), 246-264.

UNESCO (2008) . ICT Competency Standards for Teachers. United Nations Educational, Scientific and Cultural Organization.

REFERENCES