Early-stage use of The Le@rning Early-stage use of The Le@rning Federation’s learning objects in Federation’s learning objects in

schools:schools:

A report of survey and case-study resultsA report of survey and case-study results

Peter FreebodyPeter Freebody

The University The University of Queenslandof Queensland

January, 2006January, 2006

Acknowledgement and thanks to…

The principals, teachers, parents and students from the schools involved in the research

Sandy Muspratt and David McRae for collecting and analysing the data

The Le@rning Federation Contact Liaison Officers for their support

Our goalsOur goals

• recap of trial pilot findings• the results of a Field Review of the

implementation of TLF online curriculum content developed to date (surveys and case studies)

• Conclusions and ways forward: maintaining development and implementation, directing resources to professional development and evaluation of efficacy.

Review of trial pilot findingsReview of trial pilot findings(from Freebody, 2005)

The positive case:A strong prima facie case was established, from direct

observation, interview and extensive surveying, that:

• the use of TLF online curriculum content is in general supported enthusiastically by teachers, parent home-tutors and students

• the use of TLF online curriculum content motivates students to attend to and engage with tasks

• the use of TLF online curriculum content enhances students’ learning and interest in learning across a range of tasks

The CautionsThe Cautions

• Teachers need time to ensure that their selection of learning objects, from an increasingly wide range, is appropriate to needs

• Technical difficulties present ongoing frustrations to teachers and increasingly complex and consequential challenges to systems.

We speculated that …We speculated that …

• online curriculum content may operate more effectively in some task domains and for some learning purposes than in others

• these domains and purposes may not be equally distributed across curriculum subject areas

• access to online curriculum content affects the nature and efficacy of usage

The current study: goalsThe current study: goals

• to test trial pilot findings in light of improved instrumentation and sampling

• to revisit perceptions, practices and recommendations from school colleagues a year on

• to have a better sense of the range of LO-related practice in classrooms

The current study: design

• Surveys were web-administered to teachers and students (with paper-copy back-up)

• Case study sites were visited, with principals, teachers and students interviewed, and lessons observed

Percentages of survey responses

0

5

10

15

20

25

30

ACT NSW NT NZ QLD SA TAS VIC WA Missing

System

Per

cent

age

Schools (n = 186)Teachers ( n = 283)Students (n = 2518)

The case studies

Of the 17 schools• 11 urban, 3 rural, 3 remote• 7 primary, 7 secondary, 2 K-12,

1 K-8Some special features:• 5 High Indigenous• 3 Special needs / At-risk• 2 Girls’ schools• 1 Distance Education Centre

The current study: The current study: preview of main findingspreview of main findings

• There is continued strongly positive reaction from teachers and students for both learning and engagement.

• These patterns apply across all teacher and student demographics.

• Multi-level models show considerable variation within and between LOs, and some related to curriculum area.

• There are major variations in: i) awareness and usage in schools, and ii) degrees of integration.

• There is evidence of potentially new learning environments being put to ‘old’ pedagogical work

General evaluations of students

1 2 3 4 5

The learning object wasinteresting and fun

The learning object waseasy to work through

The learning object helped methink about new ideas

It helps working with a partner to do the learning object

I did not need a lot of help from myteachers to do the learning object

strongly disagree

strongly agree

Features helping students learn from LOsFeatures helping students learn from LOs

1 2 3 4 5

Sound

Colour, pictures, animation, video

Interacting with the learning object

Working at my own pace

Repeating activities until I wassuccessful

Getting feedback which told meif I was right or wrong

Getting information which told mehow to do the activity better

Ho

w h

elp

ful

we

re t

he

se

fe

atu

res

not at all helpful

extremely helpful

Teachers’ perceptions of motivational effects

1 2 3 4 5 6 7

Motivation to engage in thetask

Persistence in doing the task

Enjoyment in doing the task

Ability to collaboration withpeers in doing the task

Independence in managingand completing the task

Ho

w h

elp

ful

was t

his

learn

ing

ob

jec

t in

stu

den

ts i

n t

heir

:

extremely not at all

Teachers’ perceptions of learning outcomes

1 2 3 4 5 6 7

FACTUAL / CONTENT LEARNING

Factual content

Key processes

Label elements and parts

Define ideas and processes

CONCEPTUAL / UNDERSTANDING

Paraphrase key concepts

Explain ideas and connections

Compare and contrast

Evaluate and justify

TRANSFER OF KNOWLEDGE

Apply ideas and processes

Demonstrate applications

Construct new objects / processes

Ho

w v

alu

able

was

th

e le

arn

ing

ob

ject

in h

elp

ing

stu

den

ts in

th

eir:

extremely not at all

Clustering teachers on their assessments Clustering teachers on their assessments of learning outcomesof learning outcomes

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

Factual Conceptual Transfer ofknowledge

Learning Outcomes

Sta

nd

ard

ised

Mea

ns

Group 1 (n = 102)Group 2 (n = 82)Group 3 (n = 30)Group 4 (n = 31)

But no associations…But no associations…

• between responses and the student or teacher demographics we assessed

• that is, differing degrees of like or dislike, and nominations of key features were all distributed “randomly” across the teacher and student samples

Curriculum area effects I: Overall ratingsCurriculum area effects I: Overall ratings

• LOTE tends to rate lower on overall approval than the other curriculum areas, Literacy a little higher

• The negative effect for LOTE is statistically reliable only for “interesting and fun” and “helps me think about new ideas”

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Inte

rest

ing

and

fun

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Eas

y to

wor

k th

roug

h

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Thi

nk a

bout

new

idea

s

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Hel

ps w

orki

ng w

ith a

par

tner

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acyD

id n

ot n

eed

help

fro

m t

each

er

2

3

4

5

Inte

rest

ing

and

fun

2

3

4

5

Eas

y to

wor

k th

roug

h

2

3

4

5

Thi

nk a

bout

new

idea

s

2

3

4

5

Hel

ps w

orki

ng w

ith a

par

tner

2

3

4

5

Did

not

nee

d he

lp f

rom

tea

cher

Curriculum area effects II: Helpful featuresCurriculum area effects II: Helpful features

• LOTE tends to rate lower on ‘helpful features’ than the other curriculum areas, Literacy higher

• Statistical effects are reliable but the magnitudes are not great

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Sou

nd

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acyC

olou

r, p

ictu

res,

ani

mat

ions

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Inte

ract

ing

with

LO

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Wor

king

at

own

pace

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acyR

epea

ting

until

suc

cess

ful

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Get

ting

feed

back

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

How

to

do b

ette

r

2

3

4

5

Col

our,

pic

ture

s, a

nim

atio

ns

2

3

4

5

Inte

ract

ing

with

LO

2

3

4

5

Wor

king

at

own

pace

2

3

4

5

Rep

eatin

g un

til s

ucce

ssfu

l

2

3

4

5

Get

ting

feed

back

2

3

4

5

How

to

do b

ette

r

2

3

4

5

LOTE

Numer

acy

Scienc

eIE

CSoA

Liter

acy

Sou

nd

The specificities of the LOs IThe specificities of the LOs I

• Some LOs are rated more highly than others

BUT

• those rated highly for a particular feature are not the same set as those for a different feature

The specificities of the LOs IIThe specificities of the LOs II

• There is more variability associated with individual LOs for on-screen features (sound, colour and movement, interactivity)

THAN

• For off-screen features afforded by the LO (working at my own pace, getting feedback, etc)

e.g., Sound and getting helpful feedbacke.g., Sound and getting helpful feedback

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

So

un

d

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

Fe

ed

ba

ck

The case studies: Highlights and challengesThe case studies: Highlights and challenges

Of the 17 schools

• 1 - use of the Learning Objects (LOs) embedded into general, conventional teaching practice

• 5 - substantial use by a small proportion of staff

• 6 - some people using them from time to time• 5 - in the earliest stages of implementation

Key features: Key features: engagement and responsibilityengagement and responsibility

I love the way [the LOs] engage kids and help them in the production of their work. They are so powerful in that regard. They are a great tool in our repertoire of engaging kids. … Students are working at their own pace and … being responsible for their own learning. We can use them across the board, and find them especially helpful with our integration students (teacher, Brighton PS )

Key features: Key features: The right starting pointThe right starting point

• One of the major issues for the students I work with is finding the right starting point to enable them to learn effectively. In their normal classroom work the entry level can be just too high. Through using the objects I find that we can progress at a much faster rate through the syllabus and the sorts of content we want them to master. (teacher, Good News Lutheran School)

Key features:Key features:Independence and feedbackIndependence and feedback

• they get feedback straight away which keeps them involved and motivated. (teacher, Brighton PS)

• The independence kids can achieve is phenomenal. Variety, motivation, flexibility, ease of use, the interactivity is very important. (teacher, at-risk centre)

• They re-read — we do notice this — go back and recover. Being able to repeat is very important. It is an additional incentive, which otherwise would not be there and which our kids need, to get things right. (teacher, at-risk centre)

Key features: Key features: Summary from a dedicated userSummary from a dedicated user

• They encourage a high level of engagement from students because of their interactivity.

• The open-endedness of many of the objects stimulates further independent learning.

• The screen and interface are generally very clear and accessible.(teacher, Atherton PS)

New technologies, old pedagogies

• Pedagogy is “influenced but not determined” by LO use (case study summary notes, McRae)

• “We saw the LOs being used in ways ranging from the tightest lock-step process imaginable to unsupported open choice.” (case study summary notes, McRae)

““Like a textbook or a library book?”Like a textbook or a library book?”

“Some of the considerations that guided the thinking of some schools’ staff were: that LOs should be used for a particular educational purpose only and not as a time-filler (“if they were openly accessible they might lose some of their appeal and interest”); only the teacher should be making judgments about their use to optimise learning. We saw cases where all accessible objects were made available for staff as part of the school’s digital resource library and were indexed and delivered in a structured, pre-determined way.” (case study summary notes, McRae)

A cautionary tale

• Guttenberg and the promise of new technologies

• and Ramus, when institutions bring technologies to heel

(Ong, 1983)

Ways forwardWays forward• Preserving investment in the

implementation phase.

• Now more responsibility onto systems and colleagues in schools to take the investment to the next phase with regard to producing quality teaching and learning environments.

• That is, the outcomes are outcomes of enhanced teaching and learning, not ‘magical’ properties of the LO of itself.

ReferencesFreebody, P. (2005). Freebody, P. (2005). Does the use of online Does the use of online

curriculum content enhance motivation, curriculum content enhance motivation, engagement and learning? The Le@rning engagement and learning? The Le@rning Federation trial reviewFederation trial review. Report to . Report to Ministerial Ministerial Council on Education, Employment, Training Council on Education, Employment, Training and Youth Affairs (MCEETYA) and Youth Affairs (MCEETYA) [available at[available at

http://www.thelearningfederation.edu.au/tlf2/showMe.asp?http://www.thelearningfederation.edu.au/tlf2/showMe.asp?

nodeID=67]nodeID=67]

Images Images from Google Image.from Google Image.

Ong, W. (1983).Ong, W. (1983). Ramus: Method, and the decay of Ramus: Method, and the decay of dialoguedialogue. NY: Methuen.. NY: Methuen.