ImpaCT 2 & the continuing quest to understand ICT & Attainment

Malcolm Hunt, Becta

MIICE Conference, November 04MIICE Conference, November 04

Presentation Outline

• Key findings from ImpaCT 2

• Post ImpaCT 2 work about ICT &

Attainment

• Context issues, including ICT Pedagogy

• Summary & Conclusions

Key findings from ImpaCT 2

Pre ImpaCT 2 thinkingRESEARCH ISSUES:• Difficult to isolate any single aspect such as ICT

from other factors.• Factors are likely to be at school, organisational

and pupil levels.• All pupils have access to ICT so control group not

an option• Using a large enough sample for statistical analysis,

for a representative study• Robust/ cumulative qualitative evidence as

well to understand practice

ImpaCT 2 Overview

• Longitudinal Study over 3 years between 1999 and 2002.

• Explored the relationship between networked technologies and attainment

• Sample of 2179 pupils from 60 schools:- 30 Primary Schools- 5 Special Schools &- 25 Secondary Schools in England

ImpaCT 2 Strands & Design• Strand 1: Considered the

relationship between pupils’ use of ICT & their performance in National Tests & GCSEs as a measures of attainment.

• Strand 2: Developed and applied a variety of methods to establish how pupils use ICT, in particular out of school.

• Strand 3: Explored the nature of teaching and learning involving ICT in various settings

Measures of ICT Use:Levels of Use: from questionnairesTypes of Use: from questionnaires, pupil/teacher logs & visits by researchers

Sample: 700 pupils at KS 2, 3 and 4

Methods used :- Pupil researcher logs- Questionnaire on internet use- Special report of a key learning event with ICT-Concept mapping task

15 Cases studies with a focus on the views of pupils, teachers and parents.

How did the researchers find out how ICT affected attainment?

• 700 pupils for each key stage• Value-added data compared for ‘high ICT’ and ‘low ICT’ pupils• 13 comparisons – English, mathematics and science at KS2, 3 and 4 and – KS4 history, geography, MFL & D&T.

Baseline score

National Test/ GCSE score

Pupil B

Pupil A Expected score

Positive relative gain

Negative relative gain

Positive relative gain

Expected score

Negative relative gain

Establishing how ICT affected attainment

Pupil A has a higher test score than Pupil B, but performed worse than expected, and so has a negative relative gain score.

Pupil B, by contrast, performed better than expected and so has a positive relative gain score.

Key Findings from Strand 1

• In 12 out of the 13 comparisons, ‘high ICT’ pupils achieved on average higher value added than ‘low ICT’ pupils.

• Of the 12 comparisons 4 were statistically significant:- English at KS 2- Science at KS 3- Science at GCSE & D&T at GCSE

• In some cases the difference in average performance was equivalent to 0.2 of a National Curriculum level at KS 2/3 and 0.5 of a GCSE grade at KS4.

One level relates to around 2 years in a pupil’s development – ie they are expected to progress by the order of 0.5 of a level a year

ImpaCT 2- Getting behind the data

• Researcher judged the amount of ICT use overall to be low.

• The number of networked computers rose dramatically during the period from 1999-2001.

• Types of ICT use as well as the amount of ICT use play a part in terms of impacts on attainment/learning

• A range of other context factors play a part

Getting Behind the DataEnglish KS 2 - School by school comparisons

Mean relative gain for schools in order of ICT usage for English

Increasing ICT useLow High

• Exceptionally good teaching with less use of ICT?• The use of a particular intervention other than ICT?

Getting Behind the DataEnglish KS 2: School by school comparisons

Mean relative gain for schools in order of ICT usage for English

Increasing ICT useLow High

• Similar level of ICT use, very different relative gain score

• School T uses ICT more often than might be expected in English from the overall level of ICT use, yet English attainment is below the average

• Maybe the totality of a pupil’s experience influences achievement, rather then an isolated use in a particular subject

KS 2 English teachers identified:• The motivating effect of ICT was a common factor:

- Captivated pupils who previously found it hard to engage- Positive shift in attitude involvement in learning activities

• Types of use included word processing, publishing packages, use of the internet for research, software to aid spelling

• Higher quality outputs encouraged greater commitment to writing tasks

• Relevant software making the learning of key skills more fun (eg learning spellings)

• Increased time for reflection/ refining the content of written work

Type & nature of the Research Questions

• Need to move on from whether ICT has general educational benefit to how and why ICT works

• Need to refine our research questions and consider subsets of ICT- By subject- By ICT application eg Email, use of spreadsheets

• Focus more on the context of use

Maddux model of Maddux model of Stages in ICT researchStages in ICT research

STAGE 1: Computers produce general educational benefits

STAGE 2: Exposure to a specific application produces a specific educational gain

STAGE 3: Which and how learners interact with teaching variables when using specific aspects of ICT

Asking the ‘how’ & ‘why’ questions about ICT & Attainment

ICT & Attainment Literature Review (Cox et al 2003)

• Positive effects of specific uses of ICT on pupil attainment in almost all NC subjects

• Most substantial in Maths, Science and English, where more uses being made of different resources in these subjects

• In Maths and Science learning is most enhanced when the type of ICT use and the measures are closely related to specific skills and concepts.

‘Deeper’ uses of ICT (Cox et al 2003)

English

Science

Programming and modelling

Maths

Word processingand presentations

Simulations and modelling

Understanding andapplying concepts

Scientific reasoningand explanations

Creativity and developing meaning

Geography Simulations and modelling

Concepts andrelationships

History Multimediaresources

Understanding and historical reasoning

MFL Simulations androle playing

Specific languageskills, confidence

Subject Specific Uses Impact

ICT & Attainment Literature Review

• Impact appears strongest for ICT resources which have been embedded in teachers’ practices for a long time.

• Emerging body of evidence about specific ICT communications eg email, WWW, but the effects on attainment is not as consistent or extensive.

• The crucial component is the teacher and their pedagogical approaches

Factors in ICT Pedagogy & Factors in ICT Pedagogy & Reasoning Reasoning (Cox et al 2003)(Cox et al 2003)

ICT Pedagogy and ReasoningTeachers

subject knowledge

Access to ICT Resources

Teachers pedagogical knowledge, beliefs & values

Confidence in using ICT

Organisation & classroom

practice

Planning & preparation beyond the classroom

Factors that affect teacher uptake & use of ICT (Mumtaz 2000)

• Institutional: eg: Time, technical support, ICT resourcing

• Resources: eg: Access to computers in the classroomRange of software availableLack of training and the quality of training

• Teacher factors: eg: ICT experience, confidence/competence,Beliefs & theories about subject teaching,Knowledge & perceptions about the benefits of ICTManaging classroom activities/control Personal feelings towards ICTFear of change

Main Barriers

1. ICT Confidence/ competence

2. Access to ICT

3. Training

4. Time

5. Technical problems/support

Based on 172 practitioners

surveyed at BETT 03 and the Education

Show

0

10

20

30

% of total responses

Lack of confidence

Lack of access to resources

Lack of time

Lack of effective training

Technical problems

Lack of personal access

Age

Barrier

Barriers to ICT : Becta survey of practitioners

Broad agreement

with the literature

survey

LACK OF TEACHER

CONFIDENCE

Teacher Confidence

Lack of personal access

Access at

school

Access at

home

Technical

Problems

Fear of things going

wrong

Lack of tech.

support

Lack of teacher

competence

Lack of exploratio

n

Lack of ICT Skills

Training

Lack of Pedagogical Training

Direction of arrows represent “could be

caused by…”Red arrows show a two

way effect.

Lack of Time

Becta ICT Barriers reporthttp://www.becta.org.uk/research/research.cfm?section=1&id=3310

Personal ICT Enablers• Personal access to ICT to allow the

development of confidence and competence

• Commitment and support from departmental and senior management

• Involving whole staff in ICT decision making

• Importance and involvement of a support community of peers

• Perceived usefulness of ICT and ease of access and use

Becta ICT Enablers reporthttp://www.becta.org.uk/research/research.cfm?section=1&id=3310

ICT Standards report-

• Studied results of OFSTED inspections & QCA data on performance in national tests

• ‘ICT enablers’ that contribute to good ICT learning opportunities were:

- ICT resourcing- ICT leadership & management- ICT teaching- General School Leadership- General Quality of Teaching

Is there a ‘threshold’ effect?

Becta’s research on ICT and Standards

ICT enablers and ICT learning opportunities (primary)

%

0

20

40

60

80

100

0 1 2 3 4 5

Number of ICT enablers

Good ICT learning ooprtunities

ICT enablers and ICT learning opportunities (secondary)

0

20

40

60

80

100

0 1 2 3 4 5

Number of ICT enablers

Good ICT learning ooprtunities

%

A developing model: ICT enablers

The ‘building blocks’ of enablers that can help a school to make effective use of ICT.

ICT resources General teaching ICT teaching

Leadership ICT leadership

ICT learning opportunities

Increased attainment

Improved learning

Improved Outcomes

ICT deployed appropriately

ICT used effectively in classrooms for learning

• The ICT in schools programme

• CfT and LfT

NOF Training,HOS initiative,

CPD support materials

NCSL/Becta SLICT courses

Stages in networked ICT implementation

• Stage 1: Provision of Equipment, infrastructure and support

• Stage 2: Teaching ICT skills, often in specialist ICT lessons,

• Stage 3: Integration of ICT into the curriculum subjects, including literacy and numeracy

ImpaCT 2: Strand 2 research:

Somekh et al 2002

Pupils’ & Teachers’

Perceptions of ICT in the Home

During data collection, schools moved from Stage

1 to 2

Only a few teachers in a few schools began to move to Stage 3

TIMESCALE?TIMESCALE?

Home / School Issues • The importance of ICT use in the home and for

informal learning is increasing (ImpaCT 2 figs)- Primary School pupils: on average 3 hours a week compared to 1 hour at school- Secondary School pupils: on average 10hours a week compared to 2.5 hours at school (4x)

• Home ownership and internet access is increasing yet over a quarter of pupils did not have internet access at home. Potential disadvantage?

• Pupils have an extensive awareness of the

role of computers in today’s world (Type of equipment, use & purpose)

Students of all ages may develop ICT skills more quickly and easily than

schools anticipate

Public access to ICT in public libraries & internet cafés can lack flexibility & be

expensiveHow might schools

address this?

ImpaCT 2 ImpaCT 2 FindingsFindings Teachers recommended website for use at home & in support of subject specific learning

Emerging Findings from Becta’s Research about ICT & Attainment

Growing body of evidence of an ICT effecteg ImpaCT 2 study, SANDS Report

Beginning to appreciating the use of ICT in specific

subjects- Subject ICT traditions are importanteg ICT & Attainment literature reviews

Identifying important elements in the effective

embedding of ICT- Institutional conditions are criticaleg Barriers and Enablers research, ICT and Pedagogy research

Conclusions ICT and Standards: ensuring that we can go beyond the research questions as to ‘whether’ ICT has an impact, to ‘how’ and ‘why’ it does.

ICT pedagogy: ‘moving on from the ‘foothills of pedagogy’’ - better understanding ICT use in different subjects and with different technologies- better understanding mixed approaches blending ICT & traditional methods, including Personalised Learning

The embedding of ICT- understanding teacher engagement, CPD needs- for Institutional Development & the E Confident school

Contact detailsBectaMillburn Hill RoadScience Park Coventry CV4 7JJ 02476 41 6994

Becta Research web site: http://www.becta.org.uk/research

Malcolm Hunt: Assistant Director, Evidence and Evaluation, malcolm.hunt@becta.org.uk

ImpaCT 2 & the continuing quest to understand ICT & Attainment

Malcolm Hunt, Becta

MIICE Conference, November 04MIICE Conference, November 04

Sources:

ImpaCT 2: http://www.becta.org.uk/research/research.cfm?section=1&id=539

ICT and Attainment Report: http://www.becta.org.uk/research/research.cfm?section=1&id=3119

ICT and School Standards: http://www.becta.org.uk/research/research.cfm?section=1&id=538

Barriers & Enablers Reports: http://www.becta.org.uk/research/research.cfm?section=1&id=3310