Jornadas: Finlandia, modelo de excelencia educativa

07.04.2023 1www.helsinki.fi/yliopisto

Comparative assessment and transversal skills in

the Finnish comprehensive school: PISA, TIMSS, and Learning to Learn

Faculty of Behavioural Science/ Centre for Educational Assessment / http://www.helsinki.fi/cea/Kupiainen 2013

www.helsinki.fi/yliopisto 07.04.2023 2

Background

Assessment in the Finnish basic school

International Assessment: PISA and TIMSS

Learning to learn



Public and free of charge from pre-school year to university.

Compulsory comprehensive education for grades 1 to 9.

Practically no private schools. Parental choice of school on

side of neighbourhood school allocation.

Small – but growing – between -school differences especially in the bigger cities.


The Finnish Education System


Upper secondary education divided into two tracks. General or academic track,

acceptance based on GPA (50%). Vocational or professional track in

over 30 programs (40%). Drop-out before matriculation about 5

% + 10 % in vocational schools. Both tracks allow entrance to

universities or polytechnics through entrance exams (about 30% + 30% of the age cohort).



No systematic streaming or ability grouping though some “hidden” practises that affect school choice and class formation especially in bigger cities.

Repeating of class rare during basic education (~2%).

Pedagogy geared toward teaching of heterogeneous groups with stress on helping weaker students.

Early intervention with remedial teaching and special education.

Emphasis on student welfare: health and dental care, student welfare team, school psychologists, free lunch.


Some specifics of the Finnish school


A National Core Curriculum by the National Board of Education prepared in collaboration with an expert panel .

Municipal and school curricula based on the national document.

School and teacher autonomy in the implementation of the curriculum.

Curriculum renewed approximately every 10 years.

Goals and standards for “good performance” in each subject stated in the curriculum.

High quality text books and teacher materials.

No standardised or other national testing of all students in the basic school.


Curriculum


Need for extensive teacher (re)training due to the transit to comprehensive education in the 1970s.

Transfer of all teacher education to universities in the 1970s.

Master degree also for classroom teachers in 1979.

In 1995, also day care teachers’ education transferred to universities as a lower university degree (B.Ed.).

Lately, steps (or at least plans) toward softening the division between classroom and subject teachers in the comprehensive school.


Changes in teacher education


A sought-for track for good high school graduates, especially females without advanced courses in math and science.

Teachers’ salary relatively good for females with an academic degree and compared to the other Nordic countries.

M.Ed. valued for a variety of academic positions.

A relatively high appreciation of teachers in the society.

The relative autonomy of schools and trust in teachers indicated, for example, by the lack of any form of governmental inspection or of much testing of students.


Finnish classroom teachership


Background



Learning to learn



The education reform of the 1970s. Abandonment of school inspectorate in 1985. The gradual decentralisation of education in the 1990s. A strong commitment to equity of and in education. A high social acceptance of and trust in education. A tradition of no high-stakes tests during basic education. The first evaluation strategy for the NBE in 1995

reflecting international (OECD, EU) discussion, specifically the role of basic education in fostering lifelong learning.

Adoption of sample based assessment in key subjects in the late 1990s due to concern for growing differences in learning outcomes across the country.


Assessment – national background


The National Board of Education (NBA) Subject specific assessment at basic school. Key theoretical subjects (bi)annually, others less often. Sample based assessment for policy-level evaluation

and follow-up. Offers the assessments to municipalities for their own

evaluation purposes.

The Finnish Education Evaluation Council System level evaluations directed at diverse educational

phenomena.

The Finnish Matriculation Examination Board Upper secondary exit exam – the only high-stakes test.


National assessments


Background



Learning to learn



A growing interest during the 1990s toward assessing

not just curricular attainment but also the more general

cognitive and affective goals of education, expected to

better predict one’s ability to adapt to changes in

different domains of life, especially a lifelong and life-

wide commitment to education.


Assessment - International background


• OECD’s PISA (Programme for International Student Assessment) “assessment of knowledge and skills essential for full participation in the knowledge society” (reading, mathematics and science literacy).

• EU key competencies for lifelong learning: Communication in mother tongue; Communication in foreign languages; Mathematical, science and technology competence; Digital competence; Learning to learn; Social and civic competence; Sense of initiative and entrepreneurship; Cultural awareness and expression).

• 21st century skills: Core subjects and 21st century themes; learning and innovation skills; information, media and technology skills; life and career skills.



A three-year-cycle survey of the knowledge and skills of 15-year-olds in the OECD and other industrialised countries.

”The assessment of young people’s capacity to use in real life situations the knowledge and skills acquired at school”.

First cycle 2000 main domain READING Second cycle 2003 main domain MATHEMATICS Third cycle 2006 main domain SCIENCE

Fourth cycle in 2009 main domain READING Fifth cycle 2012 main domain MATHEMATICS the

results will be announced on December the 3rd

Questionnaires for students, principals, and parents.Faculty of Behavioural Science/ Centre for Educational Assessment / http://www.helsinki.fi/cea/Kupiainen 2013

What is PISA


PISA measures students’ understanding of key concepts in the respective areas of reading, math and science, their mastering of the key processes in them, and their ability to apply their knowledge and skills in solving authentic problems and dilemmas in the different content areas.

The tasks are jointly developed by representatives of the participating countries, taking into consideration – but not tied by – their curricula in the respective domains.

Translation, sampling and test administration are centrally coordinated to assure high validity and reliability.Faculty of Behavioural Science/

Centre for Educational Assessment / http://www.helsinki.fi/cea/Kupiainen 2013

What PISA measures?


Reading literacy Mathematics literacy Science literacy

• An individual’s capacity to understand, use and reflect on written texts, in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society.

• Text format – continuous and non-continuous texts, narration, exposition and argumentation.

• Reading process – retrieving information, forming a general understanding, interpreting, reflecting on content, form and features.

• Situations – personal, public, occupational, educational.


The three literacies of PISA


Reading literacy Mathematics literacy Science literacy

An individual’s capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to use and engage with mathematics in ways that meet the needs of that individual’s life as a constructive, concerned and reflective citizen.

Content – quantity, space and shape, change and relation-ships, uncertainty.

Process – math. language, modelling, problem solving.

Situation – personal, educational, occupational, public, scientific.




Reading literacy Mathematics literacy Science literacy An individual’s capacity to use scientific knowledge and

processes not only to understand the natural world but to participate in decisions that affect it.

Concepts – in the realms of physics, chemistry, biological sciences and earth and space sciences.

Processes – acquire, interpret and act upon evidence.

Situations (contexts) – life and health, Earth and environment, technology.




Based on interviews within the Finnish Ministry of Education and the National Board of Education, Jennifer Chung (2009) lists the ‘official’ understanding of the Finnish success in PISA as: Equal opportunities Comprehensiveness of education Competent teachers Student counselling and special needs education Encouraging evaluation Significance of education in society A flexible system based on empowerment Cooperation A student-oriented, active conception of learning.


Explaining the Finnish PISA success


Societal and historic reasons The relative homogeneity of the Finnish population and

culture, and the key role education has played in social advancement in a young nation.

Expansion of higher education from the 1960’s on – the parents of the ‘PISA-generation’.

A history of central education governance and a binding national curriculum framework safeguarding standards.

University level teacher education. Allows the recruitment of good students. Enforces and sustains the high status of teachers.

Match of the PISA framework with the Finnish curriculum Basic skills for all through a common curriculum Emphasis on application of skills and knowledge.Faculty of Behavioural Science/ Centre for Educational Assessment / http://www.helsinki.fi/cea/Kupiainen 2013

Explaining the Finnish PISA success


Success in Reading literacy expected on the basis of earlier studies.

Success in Math and Science literacy presumably due to the salient role of reading in all domains – and to similarities between PISA and the Finnish curriculum regarding the objectives, task types, and lack of tasks requiring advanced content knowledge.

Specific Finnish achievement profile in all domains with relatively better weak students – maybe reflecting early remedial and special education and the small number of students with immigrant background.


Specifics of the Finnish success

0

10

20

30

40

50

60

70

80

90

100

5 10 25 50 75 90 95

Korea

Finland

Hong_Kong_China

Canada

New_Zealand

Ireland

0

10

20

30

40

50

60

70

80

90

100

5 10 25 50 75 90 95

Chinese_Taipei

Finland

Hong_Kong_China

Korea

Netherlands

Switzerland

0

10

20

30

40

50

60

70

80

90

100

5 10 25 50 75 90 95

Finland

Hong_Kong_China

Canada

Chinese_Taipei

Estonia

Japan


READING The phonetic character of the Finnish language. All TV-programs subtitled without dubbing – an early

incentive for fast reading. A long cultural tradition for reading. A relatively language-laden (non-math) curriculum.

MATH AND SCIENCE The central role of reading in all PISA tasks. The good fit between PISA and the Finnish curriculum.

The applied nature of the tasks. The relatively low content skill requirement of the

tasks.Faculty of Behavioural Science/ Centre for Educational Assessment / http://www.helsinki.fi/cea/Kupiainen 2013

Specific reasons for success

www.helsinki.fi/yliopisto 24

Finnish students’ attainment in the OECD PISA-studies from 2000 to 2009 has been seen to exemplify the high standards and equity of the Finnish comprehensive school.

However, the results of PISA 2009 indicated a slight decline in Finnish students’ performance in two of the three measured literacy domains and a similar slight increase in between-school differences.

Latest evaluations by the National Board of Education have also pointed toward declining achievement and growing differences between schools in key subjects (Lappalainen 2011; Hirvonen 2012; Kärnä & al. 2012).

07.04.2023


From good to better, staying there – or declining?


The political nature of the OECD PISA has radically changed the policy-role of educational comparisons.

Yet, the political and cultural differences between countries have an effect on where different countries look for inspiration for change.

Whereas many Western nations might shy from the example of many of the top-performing East Asian countries – be it due to prejudice or real doubt regarding the transferability of their success – they might have rather looked at Canada and Finland for inspiration for their education systems.

25.11.2013


Comparing with and learning from others


The Nordic welfare states share much of common history and highly similar social structures (c.f., Esping-Andersen, 1990), including education systems and reform histories.

However, as reasons for the Finnish students’ recurrent outperforming of their Nordic peers are not unequivocal – is the gap due to the few differences in their respective education systems (e.g., teacher education or early diagnosing and help for learning problems) or to historical and societal differences – the knowledge gained through PISA does not easily render itself for remedies to the problems Nordic countries – now also including Finland – are currently facing in education.

07.04.2023


Comparing with close neighbours


During most of the post WWII era, the Finnish education policy and reforms have followed those of the other Nordic countries, especially Sweden.

Yet, while Finnish students came ‘on top’ in PISA 2000, the performance of the other Nordic countries was close to or even below the OECD average – a fact that might have surprised many who had not followed the development in the Nordic countries during the 1990s.

Had PISA been launched twenty years earlier, the country at top might have been Sweden…


Following Sweden ....


What seems to have happened is that the same education reforms which were implemented somewhat later and maybe with somewhat less rigour in Finland, are much to blame for the decline in educational outcomes witnessed in Sweden in the extensive NU03 assessment (Skolverket 2004).

This, in turn, may have contributed to the increase of ‘independent schools’ or the privatisation of education, itself apparently the reason for the three-fold increase seen in PISA 2009 in Sweden in the in-between school differences, setting it apart among the other Nordic countries.



While among the top performers, Finland has not been among the top in either the length of school days or in the amount of homework to substitute for the short school days.

Additionally, even if there has been growing concern regarding what many see as too much testing and teacher accountability in many countries, the Finnish success has often been explicitly tied to or explained by the lack of testing and a “mutual trust” instead of “data-driven accountability”.

How should this be interpreted?


Is Finnish education an anomaly?


Could it be that these features which have been pointed out as possible reasons for Finnish students’ success in PISA are in fact more probably responsible, together with the (relative) lull in school improvement caused by that success, for the current decline of results?

What about curricular comparative assessments?



The red line is the mean of the countries / states / provinces in the picture to be regarded as the mean for the Western developed countries.

The result is interesting when compared with the results of PISA 2009 mathematical literacy, where the Finnish students’ mean score was 541 points and the Hungarians’ 490.


TIMSS 2011 8th grade mathematicsSO

UTH

KORE

A

SING

APO

RE

TAIW

AN

HONG

KO

NG

JAPA

N

MAS

SACH

USET

TS

MIN

NESO

TA

NORT

H CA

ROLIN

A

RUSS

IA

QUE

BEC

INDI

ANA

COLO

RADO

CONN

ECTI

CUT

ISRA

EL

FINL

AND

FLO

RIDA

ONT

ARIO

UNIT

ED S

TATE

S

ENGL

AND

ALBE

RTA

HUNG

ARY

AUST

RALIA

SLO

VENI

A

LITHU

ANIA

ITAL

IA

NEW

ZEA

LAND

KAZA

KHST

AN

SWED

EN

UKRA

NIA

NORW

AY

350

400

450

500

550

600

650


Background



Learning to learn



Since 1996, educational effectiveness has been understood in Finland to include not only subject specific knowledge and skills but also the more general competences which are not the exclusive domain of any single subject but develop through good teaching along a student’s educational career.

Many of these, including, learning to learn, have been named in the education policy documents of the European Union as key competences which each member state should provide their citizens as part of general education (EU 2006) and they have been included in many national curricula.

25.11.2013



A competence refers to a complex combination of

knowledge, skills, understanding, values, attitudes and

desire which lead to effective, embodied human action

in the world in a particular domain.

Hoskins, B. & Deakin Crick, R. (2010). Competences for

learning to learn and active citizenship:

different currencies or two sides of the same coin? European

Journal of education, Vol.45, N°1, Part II pp.121-138.

Citation from http://keyconet.eun.org/about

25.11.2013


(Re)defining competence

http://keyconet.eun.org/about


Communication in mother tongue

Communication in foreign languages

Mathematical, science and technology competence

Digital competence

Learning to learn

Social and civic competence

Sense of initiative and entrepreneurship

Cultural awareness and expression

25.11.2013


Key competences - traditional and transversal


Learning to learn’ is the ability to pursue and persist in learning, to

organise one’s own learning, including through effective manage-ment of

time and information, both individually and in groups.

This competence includes awareness of one’s learning process and

needs, identifying available opportunities, and the ability to overcome

obstacles in order to learn successfully.

This competence means gaining, processing and assimilating new

knowledge and skill as well as seeking and making use of guidance.

Learning to learn engages learners to build on prior learning and life

experiences in order to use and apply knowledge and skills in a variety of

contexts: at home, at work, in education and training.

Motivation and confidence are crucial to an individual’s competence.

25.11.2013


EU Education Council in 2006


While PISA can be seen to take a step forward from (some) school subjects by looking at students’ ability to apply their skills and knowledge in everyday-like tasks, accompanied with domain-specific attitudinal questions,

Learning to learn assessment takes a step backwards to the factors leading to the learning of those subjects, i.e. trying to measure the different cognitive and affective factors responsible for learning at school and in later life – though, like PISA, measuring them at school with paper-and-pencil or computerized tasks and questionnaires.


PISA ‘vs.’ Learning to Learn

www.helsinki.fi/yliopisto

The core definition for learning to learn in the Finnish

framework can be stated as the ability and willingness to

adapt to novel tasks, or the adaptive and voluntary mastery

of learning action.

Hautamäki et al. 2002; Hautamäki, Hautamäki, & Kupiainen 2010

25.11.2013 38


The Finnish learning to learn framework


Learning to learn is seen to encompass the general educational goals that are not the sole domain or responsibility of any one subject but … are formed in good educational practice in the different

subjects … guide students’ learning and are present in school

achievement … can be understood as a precursor to lifelong learning in

indicating students’ (cognitive) propensity and (affective and volitional) willingness for learning and self-development.

Learning to learn assessment can be divided to competences (knowledge and thinking skills) learning related attitudes and beliefs.



Students' goals at school are linked to learning tasks that the teacher gives and the students are expected to accept as their own.

In this process the outer social context is to be complemented by the inner context of the self.

The solution - positive or negative - will be evaluated by the student on two ways, socially and internally.

Social comparison and achievement assessment tie the student’s response to the social system - to teachers, class-mates, and parents,

Whereas the inner evaluation ties it to the goals set by the student.

25.11.2013 40


School as the context of learning


In the assessment of learning to learn, students are given novel tasks which they are invited to accept as their own with all the motivational, goal, and aptitude-related condi tions attached, and the processes of learning (to learn) are set in motion in this acceptance.

Regardless of the knowledge or skill level of the student, the accep tance of the task (or the refusing of it) activates processes that either enhance or hinder flexible intellectual work.

With the non-curricular nature of the learning to learn tasks it is easier to see the task itself as summoning up in the student the skills and abilities needed to (start to) solve the tasks.

25.11.2013 41


… and of assessment


How do students’ cognitive competence and their learning- related attitudes develop during formal education?

To what extent do students’ general cognitive competence and their learning-supporting beliefs explain their school attainment ?

How do the relationships between these three change during students’ school careers?


General research objective

cognitive competence

attitudes & beliefs

school achievement


COMPETENCES Reasoning skills (Ross & Ross, Shayer/Piaget, Eysenck) Reading comprehension (Kintsch & van Dijk) Mathematical thinking (Demetriou, Sternberg) Questions regarding task interest and difficulty ATTITUDES or BELIEFS Motivational beliefs

Goal orientation Self-efficacy Means-ends-beliefs Agency beliefs

Academic self-concept STUDY PRACTICES

Listening in class Homework

END-OF BOOKLET QUESTIONNAIRE Effort in the assessment (not task specific)

25.11.2013


The composition of the FILLS test


A five school pilot study in 1996. National assessments at different grade levels from 1996. Studies at individual schools and municipalities. General and vocational upper secondary education studies. Longitudinal studies

From 6th grade to upper secondary in 2001-2006 From 1st grade up begun in 2007 A full cohort panel study begun in 2010 at four grade levels

Replication of the ninth grade assessment at grade 9 in 2001 in spring 2012.

Combining learning to learn with complex problem solving at three grade levels in 2013.

25.11.2013


Assessments through time


A test compiled from existing cross-curricular tests in use in Finland (learning to learn), England (learning power questionnaire), the Netherlands (cross-curricular competence test), and Spain (meta-cognition).

Result: Need for further theory-

building and instrument development (report: Kupiainen, Hautamäki & Rantanen, 2008).

25.11.2013


EU Learning to Learn Pilot Project in 2008

Cog

,23

cognA c1,48

,52

cognB c2,72

,32

cognC c3,57

,17

cognD c4,41

,50

cognE c5

,71

Aff

,81

affectiveA a1

,29

affectiveB a2

,38

affectiveC a3

,09

metaB m2

,90

,54

,62

,29

,34

GPA QUARTILE

g1

,14

metaA m1

,37,27

,25

,47

,71

-,19

-,17

-,27


Data: 7 800 ninth grade students in 82 schools in 65 municipalities.

Mean age 15.92 years.

Girls performed better than boys in all three fields of competence measured in the assessment: reasoning, mathematical thinking, and reading comprehension.

The higher the education of the parents, the better the student performed in the assessment tasks.

The between-school-differences were very small (explaining under 2% of the variance) while the between-class differences were relatively large (9 % - 20 %).

25.11.2013


National Assessment in 2012


The change between the year 2001 and year 2012 is significant.

The level of students’ attainment has declined considerably.

The difference can be compared to a decline of Finnish students’ attainment in PISA reading literacy from the 539 points of PISA 2009 to 490 points, to below the OECD average.

Girls’ attainment has declined more than boys’ in three of the five tasks.

There was no gender difference in the change of students’ attitudes.

Between-school differences were un-changed but differences between classes and between individual students had grown.

The change in attitudes—but not the change in attainment—was related to students’ home background.

25.11.2013


Declining results 2001 > 2012


The results of the assessment point to a deeper on-going cultural change, with maybe a stronger impact on the young generation.

Formal education seems to be losing its former power and the accepting of the societal expectations which the school represents seems to be related more strongly than before to students’ home background.

The school has to compete with students’ self-elected pastime activities, the social media, and the boundless world of information and entertainment open to all through the Internet.

The school is to a growing number of young people just one, often critically reviewed, developmental environment among many.

25.11.2013



The change is not a surprise, however.

It seems to follow a similar decline in student attainment registered in the other Nordic countries already earlier.

The signals of change have been discernible already for a while and the discussion following the publication of the results has led to the conclusion that it is time to open up a national discussion regarding the state and future of the Finnish comprehensive school that rose to international acclaim due to our students’ success in the PISA studies.

25.11.2013



Thank you for your interest!



Invented arithmetical operators (adapted from Sternberg 1993)

If x < y, x lag y = x + y, else x lag y = x – y. What is 13 lag 5?

Missing premises (Ross & Ross, 1979)

First fact: The water temperature of Lake Hope is 40°F.

Second fact: (five alternative choices)

Conclusion: Therefore, Lake Hope is too cold for swimming.

Control of variables (variation of Shayer et al.’s Pendulum)

Comparisons set in Formula 1 races with four variables: driver, car, tires, and track.

Hidden arithmetical operators (adapted from Demetriou & al. 1991)

6 = (5 a 3) b 4. Letter a / b stands for: addition / subtraction / multiplication / division.

Reasoning in basic mechanics (adapted from Eysenck 1966)

Pictorial tasks, e.g., three cogs with the rotation of one given, where does the last go.


The cognitive tasks


Mastery-intrinsic“To acquire new knowledge is the most important goal for me in school.”

Mastery-extrinsic “My goal is to succeed at school.”

Agency: effort

“I concentrate well enough in class.”

Control motivation

“If I fail in something I always want to find out what the reason was.”

• Self-efficacy: learning “If learning something is important for me, I know I’ll be able to learn it.”

All affective scales: Niemivirta, 2002.


The affective scales