National Assessment Program Literacy and Numeracy (NAPLAN)

Rhondda Kretschmann, Manager

This presentation will explore briefly:

• some background to the growth in testing

• testing in Queensland

• the national testing agenda

• standardised test development

• test reporting

• how schools may use test data.

Around the world, schools and school systems are under review, particularly in mature economies such as ours, where manufacturing and other key high cost industries are being exported.

There is a sense of urgency about the development of a smart economy.

Testing is an industry.

Bloomberg reports that the U.S. spends• $1.1 billion in testing

• $1.7 billion in the rapidly growing test preparation business.

Bloomberg News December 2006

The No Child Left Behind Tests alone produced approx. $500 million revenue in 2005-06.

• Wiltshire Report• 1995 – 1997 Year 6 Test• National Literacy and Numeracy Plan *• Reporting against national benchmarks• 1998 Year 3 (sample) and 5 Tests• 1999 – 2000 Years 3 (sample) 5 and 7 Tests• 2001+ Years 3, 5 and 7 Tests

* that every child leaving primary school should be numerate, and able to read, write and spell at an appropriate level (MCEETYA 1997)

History of testing in Queensland

The Wiltshire Report (1994) called for …

• Curriculum – centrality of knowledge, futures perspective, critical thinking, reporting framework, research-based

• Assessment – item bank, moderation processes, reference tests

• Early childhood – specialists, flexibility, monitoring (through running records)

• Literacy/learning difficulties – improved use of specialists, Year 2 Net, Year 6 Test

Queensland Studies Authority (QSA) developed and managed the Tests.

It was responsible for:• development of all test material• trial of test materials and test construction • desktop publishing• training of markers for the writing task• data analysis, report design and production.

The contracted provider was responsible for:• printing and distribution of all test material• scanning and data verification of test booklets• marking of the spelling & writing task• printing and distributing test results.

Tests conducted in August; reports sent to schools in November.

Queensland … until 2007

Queensland from 2008 and national priorities

• National Assessment Program (NAP)• National Assessment Program Literacy and Numeracy

(NAPLAN): cohort testing for students in Years 3, 5, 7 and 9 in literacy and numeracy; began in 2008.

• sample assessments for students in Years 6 and 10, held on a rolling 3-yearly basis; began in 2003 with Science Literacy, Civics and Citizenship (2004) and ICT Literacy in 2005.

• moving towards a national curriculum.

National Assessment Program Literacy and Numeracy (NAPLAN)

• trial in May 2006• NAPLAN Tests commenced in May 2008• replaced the state-based Literacy and Numeracy tests• for Queensland; Year 9 tested for the first time• all states complete the same tests at the same time• managed by Curriculum Corporation on behalf of MCEETYA with

the cooperation of all states’ testing managers (2008, 2009)• Commonwealth initiatives linked to results• 2010 managed by Benchmarking and Educational Measurement

Unit (BEMU) which is a sub-group of MCEECDYA (as from July 2009)

• Currently, work is focusing on the transition of all national testing arrangements to the Australian Curriculum, Assessment and Reporting Authority (ACARA)

2008 and 2009: National Assessment Program Literacy and Numeracy (NAPLAN)Curriculum Corporation is responsible for:• item development , trialling and test development• development of agreed marking standards• development of quality control standards and risk management• monitoring and reporting project progress.

Queensland Studies Authority is responsible for the provision and administration of the national tests for Qld schools. QSA coordinates:• administration• marking• reporting.

QSA also provides substantial feedback during all phases of test development, including test administration protocols.

Who is involved?

• 230 000 students• 1800 schools• Education Queensland schools• Catholic schools• Independent schools

Purposes of the tests

Overall:• improvement of student learning (gender,

language background, Indigenous)• accountability (parents, schools, systems).

Specific:• student, class and school reporting• systemic reporting• reporting against national minimum standards

(prior to 2008, reporting was against benchmarks).

Claims cohort tests include that they are useful in:

• detecting/diagnosing learning difficulties

• tracking student growth over time

• providing detailed reports to parents.

Achievement Bands

What are the achievement bands?

The results across Years 3, 5 , 7 and 9 are reported on an achievement scale from Band 1 to Band 10.

This scale represents the increasing levels of skills and understandings demonstrated in the assessments.

- Year 3: Bands 1 to 6 (Band 2 represents national minimal standard)

- Year 5: Bands 3 to 8 (Band 4 represents national minimal standard)

- Year 7: Bands 4 to 9 (Band 5 represents national minimal standard)

- Year 9: Bands 5 to 10 (Band 6 represents national minimal standard)

The nationally agreed minimum acceptable standard is needed to enable a student to make reasonable progress at school.

How are bands used?

• to report student progress at an individual level

• to report student progress for the cohort at state level

• to report in the annual National Report on Schooling in Australia.

What are standardised tests?

• Students are measured in the same way.Everyone: • takes the test on the same day • hears the same directions • reads the same passages• completes the test in the same amount of time.

• Tests are marked using common code frames, marking rubrics and analysis methods.

How are tests constructed?

Item writers refer to the test frameworks and other informing documents. (Currently, the only common documents available are the National Statements of Learning; test frameworks are draft only, awaiting alignment with national curriculum documents)

Potential test material is panelled by: • other item writers for their construct validity• curriculum experts for their content validity• systems representatives and teachers for content and face

validity.

After each of these scrutiny panels, the test items are rewritten, repanelled or rejected.

Material is trialled, and after item analysis is completed, test construction begins.

Marking

• testbooks are scanned

• data is captured using Intelligent Character Recognition (ICR)

• open-ended items are verified and responses keyed by data entry operators

• data forwarded to QSA for marking using nationally agreed marking codes

• lead markers from each state complete common marker training program

• writing task is marked by teachers using a nationally agreed criteria–based marking rubric

Literacy tested

• Reading

• Writing

• Language conventions (spelling, grammar and

punctuation)

Reading

• Students are given a magazine that contains a number of reading texts ranging from easy to more difficult

• Each text has a set of multiple-choice questions that relate to the text• key• 3 distractors

• a combination of literal, inferential and evaluative questions are presented

• a small number of constructed response questions are included.

Writing

A demand writing task is tested.Students need to know:• that demand writing is a skill• how to plan their writing• how to develop a sense of what they can do in a

specified time• how to develop effective editing routines and

strategies.

They also need to know that: • different tasks need different plans• different writers need different plans.

Language conventions

Grammar• subject verb agreement• tense• correct word/part of speech

Spelling• proof reading

o changeo find and change

PunctuationEnglish usage

Strands of the Numeracy tests

• Number

• Algebra, function and pattern

• Space

• Measurement, chance and data

A combination of multiple-choice and open-ended responses is presented.

Reporting

Three different types of reports are provided:• student report

• class report

• school report.

ANALYSE – what does it mean?

… to examine critically, so as to bring out the essential elements or give the essence of (Macquarie Dictionary)

Sample class reportThe

answers

the key

Student answers including errors.

Class and state

performance

The QSA has produced a tool that will allow you to manipulate your data

Sample school report

How can the tests help planning for learning at the class level?

• reflect on students’ performance in particular learning areas

• find patterns of misunderstandings in learning

• plan intervention programs.

How can the tests help planning for learning at the whole school level?

• analyse results at a school level

• monitor the effects of specific teaching programs.

Where do I get more information?

• QSA website: www.qsa.qld.edu.au

• NAPLAN website: www.naplan.edu.au