Quality in language assessment – guidelines and standards Waldek Martyniuk ECML Graz, Austria

Quality in language assessment – guidelines and standards

WaldeWaldekk Martyniuk Martyniuk ECML Graz, AustriaECML Graz, Austria

International codes of International codes of practicepractice

www.iltaonline.com/code.pdf www.qca.org.uk/15990.html www.apa.org/science/standards.html www.apa.org/science/FinalCode.pdf www.alte.org www.ealta.eu.orgwww.ealta.eu.org

ALTE code of practiceALTE code of practice

www.alte.org

ALTE principles of good ALTE principles of good practicepractice

ValidityValidity ReliabilityReliability ImpactImpact PracticalityPracticality Quality of serviceQuality of service

Minimum standards for Minimum standards for establishing quality profiles establishing quality profiles

in ALTE examinationsin ALTE examinations Test constructionTest construction Administration and LogisticsAdministration and Logistics Marking and GradingMarking and Grading Test analysisTest analysis Communication with stakeholdersCommunication with stakeholders


in ALTE examinationsin ALTE examinations TEST CONSTRUCTION

1. The examination is based on a theoretical construct, e.g. on a model of communicative competence.

2. You can describe the purpose and context of use of the examination, and the population for which the examination is appropriate.

3. You provide criteria for selection and training of test constructors and expert judgement is involved both in test construction, and in the review and revision of the examinations.

4. Parallel examinations are comparable across different administrations in terms of content, stability, consistency and grade boundaries.

5. If you make a claim that the examination is linked to an external reference system (e.g. Common European Framework), then you can provide evidence of alignment to this system.


in ALTE examinationsin ALTE examinations ADMINISTRATION & LOGISTICS

6 All centres are selected to administer your examination according to clear, transparent, established procedures, and have access to regulations about how to do so.

7 Examination papers are delivered in excellent condition and by secure means of transport to the authorized examination centres, your examination administration systemprovides for secure and traceable handling of all examination documents, and confidentiality of all system procedures can be guaranteed.

8 The examination administration system has appropriate support systems (e.g. phone hotline, web services etc).

9 You adequately protect the security and confidentiality of results and certificates, and data relating to them, in line with current data protection legislation, and candidates are informed of their rights to access this data.

10 The examination system provides support for candidates with special needs.


in ALTE examinationsin ALTE examinations MARKING & GRADING

11 Marking is sufficiently accurate and reliable for purpose and type of examination.

12 You can document and explain how marking is carried out and reliability estimated, and how data regarding achievement of raters of writing and speaking performances is collected and analysed.


in ALTE examinationsin ALTE examinations TEST ANALYSIS

13 You collect and analyse data on an adequate and representative sample of candidates and can be confident that their achievement is a result of the skills measured in the examination and not influenced by factors like L1, country of origin, gender, age and ethnic origin.

14 Item-level data (e.g. for computing the difficulty, discrimination, reliability and standard errors of measurement of the examination) is collected from an adequate sample of candidates and analysed.


in ALTE examinationsin ALTE examinations COMMUNICATION WITH

STAKEHOLDERS15 The examination administration system

communicates the results of the examinations to candidates and to examination centres (e.g. schools) promptly and clearly.

16 You provide information to stakeholders on the appropriate context, purpose and use of the examination, on its content, and on the overall reliability of the results of the examination.

17 You provide suitable information to stakeholders to help them interpret results and use them appropriately.

EALTA Guidelines for EALTA Guidelines for good practicegood practice

www.www.ealtaealta..eueu.org.org

EALTA Guidelines for EALTA Guidelines for good practicegood practice

• training of teachers in training of teachers in testing and assessment testing and assessment

• classroom testing and classroom testing and assessment assessment

• development of tests in development of tests in national or institutional national or institutional testing units or centrestesting units or centres

EALTA Guidelines: EALTA Guidelines: Considerations for

classroom testing and assessment

1. ASSESSMENT PURPOSE(S) AND SPECIFICATION• What is the purpose of the assessment?• How does the assessment purpose relate

to the curriculum?• Are there any test specifications?• How well is the curriculum covered?• How are the assessment purposes and

specifications made known and discussed?

2. ASSESSMENT PROCEDURES• Who designs the assessment procedures?• How appropriate are the assessment

procedures to the learners?• How is information on students´ learning

collected?• How is information on students´ learning

assessed and stored?• What efforts are made to ensure that the

assessment results are accurate and fair?• What efforts are made to promote agreement in

marking practices across teachers and schools?• What account is taken of students’ views on the

assessment procedures?

3. CONSEQUENCES• What use is made of the results?• What action(s) will be taken to improve

learning?• What kind of feedback do students get?• What processes are in place for students or

their guardians to make complaints or seek re-assessments?

• What are the consequences of the assessment procedures for classroom practices?

• What are the consequences of the results of the assessment for learners?

Documents

Quality in language assessment – guidelines and standards Waldek Martyniuk ECML Graz, Austria