Upload
hugo-potter
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Benő Csapó http://www.staff.u-szeged.hu/~csapo/
Current Trends in Educational Assessment in Hungary:
Research and Development
Prague, 26 November 2014
UNIVERSITY OF SZEGED INSTITUTE OF EDUCATION
Outline
• Current trends in educational assessment
• Educational assessment in Hungary
• Developing an online diagnostic assessments system– a large-scale project at the University of Szeged
• Further plans to develop the educational assessment in Hungary
Function of assessment: providing feedback
“If you can not measure it, you can not improve it.” Kelvin
“Measure what is measurable, and make measurable what is not so.” Galileo Galilei
• Formative and diagnostic assessment• Embedded in learning and teaching• Based on a better understanding of
learning and development• Detailed, frequent• Provides immediate feedback• Helps to determine the directions of
intervention
(2) Assessments in new contextAssessment for learning
• Innovative item formats, improved validity
• Improved objectivity and reliability
• Improved precision
• Automated scoring
• Immediate feedback
(3) New InstrumentsTechnology-Based Assessment
TBA in PISA
• 2006, major domain: science– Technology-Based Assessment of Science– international option
• 2009, major domain: reading– Electronic Reading Assessment– international option
• 2012, major domain: mathematics– Computer-Based Problem Solving– innovative domain
• 2015, major domain: science– Computer-based assessment at all domain– Collaborative problem solving as innovative domain
USA: Assessment in the “Race to the Top”
• “Adopting standards and assessments that prepare students to succeed in college and the workplace and to compete in the global economy”
• “Building data systems that measure student growth and success, and inform teachers and principals about how they can improve instruction”
Characteristics of the assessment projects in “Race to the Top”
• Two grant categories– Comprehensive Assessment Systems– High School Course Assessment
• Coalition of states• Two large coalitions received approximately
$330 million• Partnership for Assessment of Readiness for
College and Careers (PARCC) $170 million• SMARTER Balanced Assessment Consortium
(SBAC) $160 million
History and context: Two research and development communities
• Assessment centers at Budapest– National Educational Institute (-1990)
• Hungary participated in the first IEA project in the 1970s and 1980s
– After 1990: Several reorganizations– Today: Educational Authority
• International and national assessment programs
• Research and development centers at the University of Szeged – Test development projects from the 1970s– Several research groups established in the 1980s and 1990s – Today
• Research Group on the Development of Competencies (1996-)– Hungarian Educational Longitudinal Program
• Center for Research on Learning and Instruction– Developing Diagnostic Assessments
– Training of educational assessment experts (1991-)
Educational Assessments in Hungary
PIRLS TIMSS PISA
Diagnostic Assessments
Assessment of Basic Competencies
Grade
Matura(Abitur)
Diagnostic Assessment System
Characteristics of the Assessment of Basic Competences
• Reading and mathematics are the assessment domains
• Every student is measured at grade 6, 8 and 10
• The data are longitudinally connected
• Sophisticated value-added models are used
• School-level results are published (‘soft accountability’)
Developing Diagnostic Assessments
A project implemented by the Center for Research on Learning and
Instruction University of Szeged
Two phases: 2009-2011, 2012-2014
edia.hu
Online Diagnostic Assessment System
• A complex system for supporting learning and teaching• Based on the analysis of learning and developmental
processes, and the outcomes of learningSee -> framework development
• An online accessible assessment platform See -> eDia
• An item bank containing thousands of items in innovative formatsSee -> demo test for item formats
• Teacher trainingpreparing the teachers for the utilization of the system
• Developing intervention materials (e.g. computer games) for enhancing students’ skills if needed
How an assessment framework looks like?
PISA TIMSS Diagnostic
- application - content - content (literacy) - reasoning - reasoning
- application - application
A three-dimensional model for framework development
External, social expectations
GENERAL ABILITIES
Disciplinary content
Internal, psychological
LITERACY
CONTENT KNOWLEDGE – EXPERTISEPISA, three main literacy domains
Curriculum-based assessment, 1st and 2nd IEA Science and Mathematics Study
There may be unexploited
potentials here
Devising assessment frameworks:working methods and products
• Broad national and international cooperation
• Results published in 2x3 volumes (in English and in Hungarian)
• Opening theoretical chapters (international co-authors)
• Detailed standards by Hungarian expert groups
ReadingEditors
Benő Csapó and Valéria Csépe
Pyschology Literacy Discipline
Leo Blomert Wolfgang Schnotz Krisztián Józsa
Valéria Csépe Edit Katalin Molnár János Steklács
MathematicsEditors
Benő Csapó and Mária B. Szendrei
Reasoning Literacy Discipline
Terezinha Nunes Lieven Verschaffel Mária B. Szendrei
Benő Csapó Csaba Csíkos Júlia Szendrei
ScienceEditors
Benő Csapó and Gábor Szabó
Reasoning Literacy Discipline
Philip Adey Erzsébet Korom Gábor Szabó
Benő Csapó Mária B. Németh Erzsébet Korom
Result of the Framework
Development
Three volumes, published both in Hungarian and in English
May be downloaded in .pdf
The eDia platform
• The system runs at the servers of the University of Szeged
• Item builder module• Item bank unit• Test delivery module• Data bank• Data analysis module• Feedback module
The item development process
• Training the item writers (350+)– assessment theory and item writing
– content of the assessment (frameworks)
– ICT and the eDia platform
• Item writing
• Reviewing by the group leader
• Supervising by the „dimension chief”
• Piloting
• Determining the item parameters
Innovative item formats
• Text and voice for the grades 1-3• Stimulus
– text, voice– picture, animation– video– simulation
• Response capture– everything what is possible on paper, keyboard,
mouse– manipulating and moving objects on the screen, drag
and drop– touchscreen solutions
Characteristics of the Online Diagnostic Assessment System
• Serving ca. 600 000 students (grades 1-6)• Communicating with ca. 20 000 teachers• Making the service available in ca. 3 000
schools• Managing ca. 10 000 test items• Maintaining and analyzing students’ data• Providing both students and teachers with
sophisticated feedback in user friendly format
Assessments of the major domains
• Reading, mathematics and science are the major domains
• Each major domain is measured in 3 dimensions in different scales
• Large item banks are used to measure the major domains
Assessment of minor domains
• There are 16 minor domains in the system– school readiness– writing skills, musical abilities, English as a second
language– health literacy, financial literacy, visual skills, civic
competencies– combinatorial reasoning, inductive reasoning– problem solving, learning to learn, ICT literacy,
creativity– social skills, motivation
• Test batteries are prepared for the minor domains
Plans for the next phase: 2015-
• Online developmental exercises for helping slow students to catch up
• Gamification• Visualization• Integration with the regular instructional
processes• Extending to the kindergarten population
The significance of early education(The Heckman curve)
The economists’ view
The developmental psychologists’ view:
Solid foundation for later learning
Basic skillsInterestAttitudes
Motivation
Digitizing the Assessments of Basic Competencies
• Present problems– only reading and mathematics is assessed – expensive– slow feedback– difficult to interpret
• Plans for TBA– adding science– developing item banks– immediate feedback– visualizing the data to help the interpretation
Technology-based reform of the ‘matura’
• Present problems– too many elective subjects– low objectivity– no comparable scales– etc.
• Plans for the reform– technology based assessments only– 5 compulsory subjects: Hungarian language
and literature, history, mathematics, science, English (no electives)
Conclusions
• Technology-based assessments work well in kindergarten and primary schools
• Technology can be extended to other domains and age groups, the precision, objectivity and reliability can be improved
• Further work is needed to integrate the assessments into the regular classroom activities and educational processes