The PISA framework for assessment of ICT literacy

The PISA framework for The PISA framework for assessment of ICT literacyassessment of ICT literacy

Wouter van Joolingen

University of Amsterdam

Universiteit van AmsterdamGraduate school of Teaching and Learning

ICT Expert PanelICT Expert Panel

Panel Members

Irwin Kirsch (Chair), United States

Robert Kozma, United States

Chul-Hwan Lee, Korea

Jörgen Nissen, Sweden

Catherine Régnier, France

Jean-Paul Reeff, Luxembourg

Arnold Spee, The Netherlands

Wouter van Joolingen, The Netherlands

Project Staff and Counsel

Tracy Gray, United States

Marylou Lennon, United States

Jay Moskowitz, United States

Maria Stephens, United States

Claudia Tamassia, France


Why measure ICT Why measure ICT literacy?literacy?

One goal of PISA is to measure how well young– adults are prepared to meet the challenges of the

future. For many, if not most, that future will include ICT Therefore, PISA should include a measure of ICT

skills as one of its cross-disciplinary competencies.

ICT Panel members felt that the rationale for such an assessment is grounded in a number of issues of concern to

policy makers and others that include:


Why measure ICT Why measure ICT literacy?literacy?

ICT is changing the very nature and value of knowledge and information

ICT literacy in its highest form has the potential to change the way we live, learn and work

ICT literacy cannot be defined primarily as the mastery of technical skills

There is a lack of information about the current levels of ICT literacy both within and among countries


ICT literacy is the interest, attitude and ability of individuals to appropriately use digital technology and communication tools to access, manage, integrate and evaluate information , construct new knowledge, and communicate with others in order to participate effectively in society.

Defining ICT LiteracyDefining ICT Literacy


ICT processesICT processes

Access Knowing about and knowing how to collect and/or retrieve information.

Manage Organising information into existing classification schemes.

Integrate Interpreting, summarising, comparing and contrasting information using similar or different forms of representation.

Evaluate Reflecting to make judgments about the quality, relevance, usefulness, or efficiency of information.

Construct Generating new information and knowledge by adapting, applying, designing, inventing, representing or authoring information.

Communicate Conveying information and knowledge to various individuals and/or groups.


Test setup for PISATest setup for PISA

Computer-based testing– Students will work on a notebook computer

Closed web environment with test scenario’s– Simulated “real ICT-world”

Small web Simulated responses from the system and the

external world


SCENARIOS TASKS

Technology Environments Web Desktop E-learning simulations

Contexts Personal Public Educational Occupational

Processes Access Manage Integrate Evaluate Construct Communicate

Functionality Representations

Word Processing E-mail Presentation DatabaseE-commerce SpreadsheetConcept Mapping Browser (includes search engines)

Text Images Numbers Graphics

Building ICT Tasks


A possible scenarioA possible scenario

Your friend, Jim, has a new hobby, collecting stamps. He just started it, and is still a beginner on this topic. His birthday is approaching and you decide you want to give Jim a book on stamp collecting, that matches his level of expertise. You don’t know anything about stamps, so you start searching for a suitable book.


Some considerationsSome considerations

To find good books, learn something on the topic yourself

Use smart search strategies to identify central topics

Evaluate reliability of reviews and different bookshops (including 2nd hand) and compare deals.

Need to make more explicit than in “natural” settings.


ScoringScoring

Access

Search strategies – key words used

Recycling key words, sites used.

Go halfway in the buying process

Manage Organization of reviews

Integrate Linking reviews, books and other knowledge

EvaluateAssess trust in reviews and shops – sources used in task products

Construct Build ordered lists of findings

CommunicateMake deal with on-line shop

Present findings to others


The frameworkThe framework

Allows for the construction of an assessment instrument that– Does justice to many aspects of ICT literacy– Allows for testing in realistic situations– Goes beyond self assessment– Provides an objective account of an

individual’s ICT skills and knowledge.


PlansPlans

Feasibility study– Try-out with small number of test scenarios– USA, Australia, Japan– Data collected – analysis in progress

Test development, field trial (2004-2005)Inclusion in PISA 2006

– At least 15 OECD countries