Embed Size (px)
Citation preview
This article was downloaded by: [University of North Texas]On: 30 November 2014, At: 17:40Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
The Curriculum JournalPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/rcjo20
Research into learning to learnthrough the assessment of quality andorganization of learning outcomesBenő Csapó a
a Department of Education , University of Szeged , ResearchGroup on the Development of Competencies, Hungarian Academyof Sciences, HungaryPublished online: 28 Jun 2007.
To cite this article: Benő Csapó (2007) Research into learning to learn through the assessmentof quality and organization of learning outcomes, The Curriculum Journal, 18:2, 195-210, DOI:10.1080/09585170701446044
To link to this article: http://dx.doi.org/10.1080/09585170701446044
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Research into learning to learn through
the assessment of quality and organization
of learning outcomes
Ben}o Csapo*Department of Education, University of Szeged, Research Group on the Development of
Competencies, Hungarian Academy of Sciences, Hungary
This article examines the problem of learning to learn from the perspective of research on the
organization and quality of students’ knowledge. This approach is based on the assumption that
students’ learning competencies can be studied through the analysis of the outcomes of schooling.
The article synthesizes findings of a long-term research programme assessing the development of
components of propositional and procedural knowledge, and relationships between them. A model
was devised classifying indicators of school outcomes on four levels: (1) teachers’ grading;
(2) objective knowledge tests; (3) indicators of quality of knowledge (application, understanding);
and (4) higher order thinking skills. A set of tests and questionnaires were administered to Year 7
and Year 11 students in two consecutive projects; then, relationships between them and differences
between the two cohorts were analysed. As results indicate, a number of points in schooling should
be changed in order to improve schools’ impact on learning to learn, for example, inadequate
feedback given by grading, and shifting attention from reproductive to productive learning. The
instruments developed in this programme assess science application, science misconception,
mathematical understanding, historical reasoning, and inductive, deductive, spatial and critical
reasoning. They can also be utilized for monitoring learning to learn.
Keywords: Assessment; Learning to learn; Organization of knowledge; Quality of learning
outcomes; Thinking skills
Introduction
It is generally accepted that learning to learn is a complex concept (Hautamaki et al.,
2002). There is no single interpretation of it, as several views exist on how it should be
interpreted. Two main types of interpretation include a narrower and a broader one
(Waeytens et al., 2002). In this article I accept the broader interpretation and argue
*Department of Education, University of Szeged, Pet}ofi sgt. 30–34, 6722 Szeged, Hungary.
Email: [email protected]
The Curriculum Journal
Vol. 18, No. 2, June 2007, pp. 195 – 210
ISSN 0958-5176 (print)/ISSN 1469-3704 (online)/07/020195–16
ª 2007 British Curriculum Foundation
DOI: 10.1080/09585170701446044
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
that, in order to make educational systems more supportive, the learning processes in
which students are engaged should be modified at a number of points.
One of the most striking experiences of schooling is that students learn a lot at
school: they master a large body of knowledge but they are unable to apply it outside
their educational contexts. In many cases, the application of knowledge is limited to
the school subject in which it was acquired. This problem is not at all new. During the
long history of formal education, a perennial question has persisted: ‘how should we
teach our students in schools in order to provide them with valuable and applicable
knowledge?’ The Non scholae, sed vitae discimus! slogan has remained an ideal that has
never been reached, either in ancient or modern times. In modern societies, where
lifelong learning is a demand, the vision concerning the application of knowledge
should be extended to another dimension: schools are expected to prepare students to
be successful in a rapidly changing social environment; in brief, to apply their
knowledge in the unknown future.
The concept of ‘learning to learn’, as it is known in the literature, especially in the
early formulations of the idea, implies that students should learn more. On the other
hand, everyday experiences as well as rigorous investigations indicate that students
spend a large amount of time learning, and they are able to reproduce textbook
knowledge if it is measured by knowledge tests strictly based on curricula. Therefore,
in order to make students better at learning, the outcomes of school learning have to
be analysed, and those features of school practice which prevent schools from
accomplishing their expected mission in this domain have to be identified.
This approach was adopted by a group of researchers at the University of Szeged
when a long-term research programme was launched in the early 1990s with the aim
of a comprehensive evaluation of the outcomes of school learning. It began with
preliminary investigations and continued with two complex projects entitled ‘School
knowledge’, with major data-collection waves in 1995 and 1999. This was followed
by another wave to assess the development of abilities and skills most relevant in the
context of lifelong learning (1997–2002). These cross-sectional assessments were
carried out on nationally representative samples involving 5–10,000 students per
cohort.
The results and implications of these studies initiated further studies, with more
focused investigations of some crucial issues related to the quality of learning
outcomes. Finally, a long-term longitudinal study was launched in 2003 on nationally
representative samples in three cohorts to investigate causal relationships between
the characteristics of early development and success in later school learning. Although
the whole research programme is not directly aimed at studying the issue of
learning to learn, this body of interrelated investigations allows us to look into this
paradigm.
The present article provides an overview of the main findings from the perspective
of learning to learn. First, the projects related to the organization and quality of
students’ knowledge will be reviewed, and since these projects applied a complex
system of variables, they will be described in detail. Then, the most recent projects
and work in progress will be summarized, indicating how the focus of research has
196 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
shifted to integrate further questions of learning to learn. Finally, implications will be
discussed.
Theoretical and research background
Two main types of indicators signal problems with students’ abilities to learn. One
obvious sign is that children do not learn, or they do not learn enough. This kind of
problem is mainly related to the affective domain, mostly to interest and motivation.
The other, mostly cognitive problems are experienced when students actually learn—
in terms of spending a lot of time with learning—and they are able to reproduce what
they have learnt in certain conditions, but the result of learning is inappropriate; the
knowledge is quickly forgotten, good for nothing, or way below expectations. The two
types of problem are usually in causal relationship with each other. At the beginning
of schooling, most children are naturally interested in learning and motivated to
master new knowledge; however, if they are regularly forced to learn something
irrelevant and incomprehensible for them, they gradually lose motivation and interest.
Thus, the first problem that triggers research into the issue of learning to learn is
that students do not learn well. The problem with their knowledge—that is not trivial
and deserves scientific analysis—is not related to quantity of knowledge, but a
problem of quality. This issue has been conceptualized in several ways.
The research presented here was mostly influenced by theoretical frameworks
identifying two types of knowledge, procedural (abilities and skills) and propositional
(declarative, e.g. facts, information) knowledge, and the examination of their relation-
ships. It is generally assumed that one of the main functions of abilities and skills is
processing (e.g. analysing and organizing) declarative knowledge. Therefore, their
development is a precondition of successful learning and their low level hinders
learning and results in poor quality of knowledge.
Several research paradigms offer guidelines for the identification of the most
important skills and abilities which play a central role in organizing knowledge. The
psychometric tradition (with intelligence research at its core) emphasizes general
cognitive abilities (for a synthesis see Carroll, 1993). From this paradigm, those
general abilities that were supposed to have a major contribution to knowledge
accumulation and knowledge utilization in new situations were taken into account:
inductive reasoning and analogical reasoning. These abilities were studied in our
research programme and their development and functions were assessed in several
contexts. The development of inductive reasoning was assessed on smaller (Csapo,
1997a) and countrywide representative samples (Csapo, 2001) and in projects
examining the organization of knowledge.
Another important source are the Piagetian and Neo-Piagetian traditions; from
these a constructivist view of learning and some areas of operational reasoning was
adopted. Those operations that, under normal circumstances, appear at an early stage
of development are particularly important. Such operations are class inclusion,
seriation, classification, multiple classification and combinatorial operations and, in
the later stages of development, operations of propositional logic. These operations
Quality of learning outcomes 197
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
are essential in identifying elements of learning materials, discovering relationships
between them, and understanding the meaning of complex sentences; therefore, if
they are not developed well enough, meaningful conceptual learning is impossible.
(Several projects carried out in this paradigm are published in an edited volume by
Demetriou et al., 1992).
Some models attempt to integrate the results of these two paradigms and include a
third influential paradigm, the information-processing approach. One of the most
elaborated and empirically tested models is developed by Demetriou (2004).
One obvious implication of these relationships between procedural knowledge and
learning is that skills and abilities, essential for successful learning, should first be
enhanced, if they are not developed well enough when they are to be used. Many
developmental programmes were inspired by this simple relationship, acknowledging
the importance of preliminary knowledge in terms of abilities and skills. Improving
thinking and learning skills has long been considered a central mission of schooling
(see Glaser, 1984; Bransford et al., 1985); however, empirically tested methods rarely
progressed beyond the experimental phase.
Most research and developmental programmes concerning teaching and learning
thinking skills (see Hamers & Csapo, 1999) aimed at providing students with the
skills necessary for further learning. From our point of view, content-based training is
especially relevant, since it uses teaching materials for composing developmental
exercises. These techniques of devising training materials are often called enrichment,
infusion or embedding. Besides developing students’ crucial general abilities and
thinking skills, they improve understanding of the content of teaching materials
(Csapo, 1990, 1999). For example, the CASE programme (Cognitive Acceleration
through Science Education—see Adey & Shayer, 1994; Adey, 1999) utilizes science
materials for improving students’ general cognitive abilities.
Studies on improving students’ general abilities and some specific thinking skills
that play critical roles in school learning have direct consequences for learning to
learn. Our experiments aiming to develop students’ operational abilities (e.g. logical
and combinatorial reasoning) in the context of teaching disciplinary knowledge
(grammar and science in 4th grade, chemistry and physics in 7th grade) have clearly
shown that training empowers students with the skills necessary to comprehend the
structure of learning materials. Developing reasoning skills ‘on the spot’, in the
context of learning disciplinary knowledge, removes the obstacles that hinder
successful learning (Csapo, 1990, 1992, 1997b, 2003). For example, improving
students’ logical abilities in specific exercises using statements of the teaching
material helps them to understand sentences otherwise incomprehensible to students
whose logical abilities are not developed enough without training (Vidakovich, 1996).
In the context of learning to learn, ‘dynamic’ experimental studies complement
‘static’ research studying the structure and quality of knowledge in a more direct way.
This latter approach has often been conceptualized by distinguishing two different
kinds of learning that result in different quality of knowledge. The first is more
natural, often incidental, and it takes place in authentic contexts, based on interest
and intrinsic motivation, usually outside school or in other formal instructional
198 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
settings. Although the knowledge acquired in these circumstances may be limited, it
is immediately applicable in the appropriate context. The second type is school
learning, when students are supposed to master a large volume of knowledge, but
they go through the instructional processes along a prescribed agenda, independent of
their interest, motivation and, very often, of their prior knowledge.
A number of terms are used for this problematic process (e.g. rote learning), and its
compartmentalized result consisting of isolated elements called scholastic or inert
knowledge (Bereiter & Scardamalia, 1985). This is far from the ideal of meaningful
conceptual learning building knowledge on proper prior knowledge and deep
structural understanding.
In our research we coined the term school knowledge for the outcomes of school
instruction. The aim of the programme was to examine what school knowledge
comprises in reality and where real learning processes on the rote learning—
meaningful learning scale can be placed. Results have been presented at several
conferences (Csapo, 1996, 1997c) and published in journal articles and in two
comprehensive volumes in Hungarian (Csapo, 1998a, 2002). In this article
I summarize the results of this research and draw its conclusions for learning to learn.
Studies on the quality and organization of students’ knowledge:
framework, research questions and overall design of projects
These projects intended to devise methods and instruments to characterize the
quality of school learning and to identify areas where improvement is most needed;
therefore, a large system of variables was applied. We developed a framework that
organizes the indicators of the outcomes of schooling into a system of variables that
describe students’ knowledge at four levels.
At the first level, grades given by the teachers were collected. These are indicators
of students’ achievements that both students and their parents first face. Depending
on the general culture of schooling, the preparation of teachers and the evaluation
technique available, the reliability of grading practice may vary to a great extent. It
may depend on local school values and the subjective value judgement of the
individual teachers. This is the most important feedback that orients students in their
learning, shapes their self-concept, attitudes and motivation, and influences parents’
decisions concerning their children’s future. Therefore, the first research question
was related to the quality of feedback provided by teachers’ grading. This was
conducted by correlating grades with other variables.
At the second level, objective knowledge tests of school subjects were applied.
Since the same tests were administered to all participants, these data resulted in
indicators that are independent from the local variety of school expectations, value
systems and teachers’ subjectivity. These tests were based on school curricula and
mapped the knowledge represented in textbooks and other learning materials into
tests. Since they asked questions in the context of a given subject in the same way
in which students are usually tested and are expected to demonstrate their knowledge
at school, their results are indicators of how well schools meet their external goals.
Quality of learning outcomes 199
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
The second research question is related to the knowledge tests: how much subject
matter students master at school and how it is distributed over students and schools
(i.e. how large are the differences between students and schools).
At the third level, tests go beyond the immediate reproduction of subject matter
knowledge. They examine the quality of their knowledge: how consistent it is, how
well students understand what they have learnt, how they can apply it in new contexts
that are different from where it was mastered. These tests were constructed on the
basis of general claims concerning schooling, stating that schools should provide
students with knowledge broadly applicable beyond school in private and later
professional life. In relation to this issue, the third research question examined the
quality of knowledge mastered at school.
At the fourth level, students’ general cognitive skills (and some learning-related
affective attributes) were assessed. First of all, those thinking skills were included that
play an important role in learning, understanding, knowledge transfer and application
processes, and that are important in self-development. For the two latter levels,
theoretical resources and empirical results of modern cognitive research were utilized.
To place the problem into a developmental context, the assessments were carried
out in two age groups. In the framework of a cross-sectional design (beyond the
subject matter specific knowledge tests), the same tests (the third and fourth levels)
were administered to Year 7 (age 13) and Year 11 (age 17) students. These two
cohorts represent crucial points of schooling, since they precede the closing phase of
primary and secondary education by one year. During the four years between these
two points students are supposed to master a great deal of learning materials, and
schools have a lot of time and opportunities to develop students’ applicable
knowledge and general skills. The fourth research question is related to changes in the
quality of knowledge and development of general skill; it can be answered by
comparing the results of the two age groups.
Both projects—the one in mathematics and sciences and the other in the
humanities—were carried out by a research team, where each member was
responsible for certain areas—developing the instruments and performing in depth
analyses of the given area. Beyond the main research questions listed above, a number
of other particular issues were also examined. The main publications that resulted
from these specific analyses will be cited in connection with the instruments.
Methods
Participants
The samples for the projects were drawn from Szeged and its metropolitan area.
Szeged (about 150,000 inhabitants) is one of the largest towns in the south-east of
Hungary, with diverse primary and secondary schools and higher educational
institutions. National surveys show that student achievements in the schools of
Szeged and its neighbourhood are close to the national average and sometimes
surpass it.
200 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
For the study of age differences, samples were selected from two age groups. The
Year 7 sample was drawn from the elementary schools and it was considered
representative for the whole population of this age group. The Year 11 sample was
proportionally selected from two types of four-year secondary schools (grammar
schools and comprehensive schools) that prepare students for entry to higher
education. Since vocational school students were not involved in the survey, the 11th
grade sample represents only students attending four-year high schools. The number
of students in each sample exceeded 500.
Sampling units were whole classes proportionally selected from the schools of the
area for a proper representation of different school types. School classes as units were
also used in some data analyses. Sampling and data analysis procedures were identical
in the two consecutive studies and in both cohorts.
Instruments and design of data collection
Due to limitations in educational research (students can be tested only in a few areas,
in a relatively short period), data were collected in two phases using the same model.
One focused on mathematics and science (MS); and another examined humanities
and social studies (HS). Some instruments and questionnaires were used in both
surveys in order to check the stability of findings and the consistency of results,
whereas other instruments were specific for the given project. In the following
description of the instruments the acronyms MS and HS indicate the projects in
which instruments were applied in the survey. Figure 1 summarizes the system of
variables for the MS study, while Figure 2 visualizes the HS study. Besides the tests, a
general questionnaire was also administered to students to collect an extensive set of
background data.
Grades (MS, HS). In Hungary a five-point scale is used for grading students’ school
achievements: 1 means ‘fail’ and 5 is ‘excellent’. At the end of each semester, students
Figure 1. The system of variables in the mathematics and science (MS) project
Quality of learning outcomes 201
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
receive a summative grade in all school subjects that reflects their global achievement
during that period. These grades were collected in all main subjects (mathematics,
chemistry, physics, biology, Hungarian literature, Hungarian grammar, history and
foreign languages).
Knowledge tests. These tests were summative measurement tools covering students’
knowledge mastered during a school year. They were compiled by independent
experts and were administered at the end of the school year. Mathematics (MS),
chemistry (MS), physics (MS), biology (MS), Hungarian literature (HS), history
(HS) and English as a foreign language (HS) tests were administered in the surveys.
The main principle of constructing knowledge tests was that they measured students’
knowledge in the same contexts where they mastered it.
Understanding and application. These instruments were designed to assess the quality
of knowledge: how well it was understood and could be applied. The tests were
designed by utilizing a number of recent cognitive issues (e.g. conceptual change,
science misconceptions, history misconceptions, modes of representation, transfer of
knowledge). At this level we administered a Science application test (MS) validated and
used previously (see B. Nemeth & Csapo, 1996; Csapo, 1997a; B. Nemeth, 1998,
2000). It consists of thirty-five items that can be solved by applying knowledge of
everyday problems learned in science subjects. The Mathematics comprehension test
(MS) contains problems that can be solved only if students have a deeper
understanding of corresponding areas of mathematics (Dobi, 1998). A Science
misconceptions task battery (MS) was used to study students’ conceptual development
and conceptual consistency; from among the seven tasks of the battery, two were
borrowed from the literature and five were devised for our studies (Korom & Csapo,
1996; Korom, 1997a, 1997b, 1998, 2000). As these different tasks do not cover a
consistent body of knowledge, they do not form a single test. However, relationships
between the achievements on these tasks were strong enough to represent them as a
single variable in further computations.
Figure 2. The system of variables in the humanities and social sciences (HS) project
202 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
The History-related reasoning test (HS) comprised twelve tasks and was constructed
according to similar principles (Szebenyi & Vass, 2002). These instruments were
designed to examine students’ misconceptions and were evaluated both qualitatively
and quantitatively. A questionnaire was administered to examine students’ text
production strategies (HS) and participants wrote a composition (Text creation test—
HS) that was analysed by both qualitative and quantitative techniques (E. K. Molnar,
2000, 2002). An English language competence (HS) test measured students’ language
skills in lifelike contexts (e.g. a letter-writing task; see Bukta & Nikolov, 2002). The
Visual arts test battery (Spatial reasoning test, Visual environmental test, Art taste
test—HS) includes a number of tasks tapping into spatial abilities and recognition of
styles (Karpati, 2002).
Higher order thinking skills, motivation, self-concept. The Inductive reasoning test (MS)
consists of a verbal analogy, a number analogy and a numbers series sub-tests
(Csapo, 1997a, 1998b). In the HS project only verbal analogy sub-scales were used.
The Deductive reasoning test (MS) involves the basic operations of propositional
logic (Vidakovich, 1996, 1998). The Probabilistic/correlative reasoning test (MS)
examines how children perceive probabilistic relationships (Ban, 1998), whereas the
Critical thinking test (HS) requires critical and evaluation-type reasoning (L. Molnar,
2002).
Background variables. A questionnaire about participants’ parental education, self-
esteem, future plans and other factors that may influence their achievements was
administrated to each sample (MS, HS), while two further questionnaires enquired
into students’ motivation and academic self-concept (Jozsa, 2002).
Results and discussion
The two projects resulted in a rich description of the quality of school knowledge.
I will summarize some of the relevant findings according to the research
questions.
Immediate feedback that orients students’ learning: characteristics of teachers’ grading
practices
This included the way students learn to learn at school, that their study habits change
over time and that their learning-related attitudes develop in strong relationship with
feedback received from their teachers. Since grades play a determining role in
students’ careers, teachers’ grading practices were analysed at three levels.
In the student-level analyses, the correlations were computed between students’
grades in school subjects and other variables. These analyses showed what the real
basis of teachers’ grading was. The grades correlated with achievement test results to
varying degrees: they were acceptably high in mathematics and sciences (0.5–0.6),
but quite low in social studies; the lowest correlation was found in history (0.28) and
Quality of learning outcomes 203
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
literature (0.25) in Year 11. Much lower, in many cases insignificant correlations,
were found with other cognitive variables, including ones indicating understanding or
applicability of knowledge in the same area; for example, in Year 11 the correlation
between chemistry grade and science application test was low (0.23—see B. Nemeth,
1998), whereas no relationship was found between history grade and historical
reasoning (70.05—see Szebenyi & Vass, 2002).
Between-class analyses indicated large differences between school subjects
(correlations were computed between means of classes: see Table 1). For example,
Year 11 mathematics teachers grade their students very similarly, as the correlation
between the class means of mathematics grade and mathematic test result is high,
while in other subjects (e.g. in Year 7 history and in Year 11 literature) teachers’
grading is less consistent.
Within-class correlation coefficients (correlations were computed for every single
class) were much lower than the above ones. These data indicate that many teachers
lack a professional value system that would give a firm basis for evaluating students’
knowledge.
Distribution of students’ subject matter knowledge
Results of knowledge tests indicate that students mastered a large portion of the
materials represented in their textbooks; the means usually varied around 50 per cent.
These results confirm the hypothesis that students learn what they are expected to
learn.
On the other hand, there were large variations between students, and the way they
differ from each other indicates several problems. Large differences were found
between the means of school classes, and also between students according to the
level of education of their parents and other indicators of family socio-economic
status. These results were later confirmed by the PISA (OECD, 2004) and inter-
national comparisons indicate that these types of differences are especially high in
Hungary, therefore problems behind this phenomenon are more serious than in other
countries.
Table 1. The correlation coefficients between the means of grades and knowledge test results
Subject Year 7 Year 11
Biology 0.67 –
Physics 0.59 0.61
Chemistry 0.63 –
Mathematics 0.75 0.92
English as a foreign language 0.52 0.58
Literature 0.46 0.25
History 0.27 0.42
Source: Data are summarized from Csapo, 1998a, 2002.
204 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
The major impact of family background means small school impact, and from the
point of view of learning to learn, this indicates that students’ learning is probably less
shaped by schools than by other environmental factors.
The quality of students’ knowledge: application, understanding and thinking skills
A detailed analysis of the other cognitive test revealed further problems that can be
related to students’ inadequate learning. For example, the science application test
indicates that students can hardly interpret some basic scientific facts in everyday
contexts (B. Nemeth, 1998). School learning fails to replace misconceptions with
sound scientific knowledge (Korom, 1998; Szebenyi & Vass, 2002). Mathematics is
not well understood and students cannot solve simple problems if they go beyond the
well-practised routines, even if they are simpler than the regular textbook problems
(Dobi, 1998). Students cannot solve tasks embedded in communicative situations,
although they perform well on vocabulary and grammar tests (Bukta & Nikolov,
2002).
Analyses that studied the relationships between the cognitive variables, for example
cluster analyses, indicated that students’ knowledge is divided into two independent
bodies: one mastered and applied within school learning contexts and another one
originating from everyday experiences. The first one is correct from an academic,
disciplinary point of view, but not usable outside the school context, and the other is
more pragmatic but often leads to false generalizations; transfer is limited in both
cases. Correlational analyses also revealed the major role inductive and, especially,
analogical reasoning play in learning.
These results indicate a number of inadequacies in students’ learning that may be
related to inefficiencies in school instruction that condition students for memoriza-
tion and reproductive learning. These indications of poor learning strategies by
Hungarian students were later also confirmed by the PISA study (OECD, 2003).
The impact of schooling on the development of general abilities and thinking skills
The most striking results of these projects were the ones that revealed how slowly
general cognitive skills develop; some of them even decrease during the years of
schooling. The results of some tests (identical for the two cohorts) are summarized in
Table 2.
The proportion of changes over the four years of schooling varies broadly from test
to test. In general, development is very slow. In two cases (correlative reasoning and
visual environment), changes are negative. The more students study at school, the
less they are able to recognize and accept probabilistic relationships.
Effects of schooling on affective variables
Another negative tendency was observable concerning affective variables. Three sets
of data were available in these studies on affective variables. The background
Quality of learning outcomes 205
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
questionnaire contained questions on how much students liked to learn each subject.
In most cases, students in Year 11 showed more negative attitudes than students in
Year 7; English as a foreign language was the only exception. This problem was later
examined with the same technique over a broader age range (from Year 5 to Year 11),
and the results were similar: physics is the least, chemistry the second least popular
subject (Csapo, 2000). Both mathematics and biology are more popular, so there
must be something wrong with the teaching of these two science subjects. The
negative attitudes towards chemistry and physics indicate that students are forced to
learn them without finding pleasure in them. Such practices are damaging for
learning to learn; therefore, these distracting features should be identified and
eliminated.
In the HS study, a mastery motivation test was administered to both cohorts. The
same negative tendency was observed: Year 11 students were consistently less
motivated than their younger schoolmates (Jozsa, 2002).
General conclusions and implications for learning to learn and areas
for further research
In our interpretation, the problems addressed by the issues of learning to learn are
related to the quality, and not to the quantity, of the knowledge students master at
school. On the other hand, these problems can be conceptualized so that the quality
of knowledge can be characterized by quantitative measurable indicators. Such
indicators have to be related to the understanding, organization, transferability
and applicability of knowledge. Furthermore, learning results in better knowledge
if it is supported by an elaborated affective system—which can also be characterized
by quantitative indicators. This approach implies that learning competencies are
not mastered in a single straightforward way, but that they are shaped by a broad
range of developmental influences to which learners are exposed during their
activities in and out of school. In the projects presented here, we have identified a
number of problems related to the quality of learning experiences and the accuracy
Table 2. Performance of the two age groups on some tests (in % of the maximum test score)
Test Year 7 Year 11
Science application 33 55
Misconception 50 64
Mathematics understanding 34 47
Inductive reasoning 45 64
Deductive reasoning 61 76
Correlative reasoning 57 54
Spatial reasoning 49 53
Visual environment 74 68
Critical thinking 33 40
Source: Data are summarized from Csapo, 1998a, 2002.
206 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
and adequacy of feedback students receive from their school environment—
predominantly, their teachers.
Since the problems which hinder meaningful conceptual learning that results in
well-understood transferable knowledge are manifold, there is not a single solution
to treat them. Teaching and learning processes in school must be changed in a
number of ways. One major problem is that skills, especially students’ general
information processing skills, are not practised in the context of the learning of school
subjects. Therefore, the experiments on developing thinking skills using the embedded
approach have been extended to other areas. For example, analogical reasoning was
developed in biology materials, and the effects of training were also detected in terms
of deeper understanding and better mastery of disciplinary knowledge (Nagy, 2006).
Further experiments are in progress to improve younger students’ inductive reasoning
skills.
Further research is needed into the application of knowledge in new situations.
Based on the models of knowledge transfer and the interpretation of complex
problem solving (CPS) as application of existing knowledge to novel situations,
assessment instruments were devised that can be applied in the context of teaching
disciplinary knowledge. Such a test battery was developed by using science and
mathematics materials (Gy. Molnar, 2001, 2006). As for improving CPS, experi-
ments are being prepared for problem-based learning.
Another set of problems concerns the organization of propositional knowledge,
concept development and misconceptions. Previous assessments were extended and
further instruments were devised to monitor concept development and to promote
conceptual change (Korom, 2005). Further research was initiated to apply concept
maps both for assessing and representing students’ knowledge structures and to
devise training exercises to make their learning more meaningful (Habok, 2004). An
experiment is presently in progress in this area.
As was realized at several points in the research, a major difficulty is related to
the evaluation of students’ knowledge. To solve this problem a number of instru-
ments have been devised to assess the quality of students’ knowledge and to monitor
their development. To improve the feedback students receive from their teachers,
we should further differentiate and enrich the evaluation technique and instru-
ments; thus, a variety of formative and diagnostic tests is needed. Furthermore,
since teachers are the most important resources of school instruction, better
preparation of teachers and more elaborated knowledge about student learning is
required.
Finally, empirical studies have resulted in a theoretical generalization that
synthesizes different approaches to identify and classify the goals of schooling and
the organization of knowledge (Csapo, 2004).
Acknowledgements
The work presented here is part of a long-term project of the Research Group on the
Development of Competencies (MTA-SZTE Kepessegkutato Csoport). The data
Quality of learning outcomes 207
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
collection was supported by the Hungarian National Science Foundation (OTKA
T018577, T030555 and 46659).
References
Adey, P. (1999) Thinking science: science as a gateway to general thinking ability, in:
J. H. M. Hamers, J. E. H. van Luit & B. Csapo (Eds) Teaching and learning thinking skills
(Lisse, Swets & Zeitlinger), 63–80.
Adey, P. & Shayer, M. (1994) Really raising standards: cognitive intervention and academic achievement
(London, Routledge).
Ban, S. (1998) Gondolkodas a bizonytalanrol: valoszın}usegies korrelatıv gondolkodas [Reasoning
about the uncertain: probabilistic and correlational reasoning], in: B. Csapo (Ed.) Az iskolai
tudas [School knowledge] (Budapest, Osiris Kiado), 221–250.
B. Nemeth, M. (1998) Iskolai es hasznosıthato tudas. A termeszettudomanyos ismeretek
alkalmazasa [School knowledge and usable knowledge. Application of science knowledge],
in: B. Csapo (Ed.) Az iskolai tudas (Budapest, Osiris Kiado), 115–138.
B. Nemeth, M. (2000) A termeszettudomanyos ismeretek alkalmazasa [Application of science
knowledge], Iskolakultura, 10(8), 60–68.
B. Nemeth, M. & Csapo, B. (1996) Learning for the schools or learning for life?, poster presented at
‘The Growing Mind’, International Conference in Honour of the Centennial of Jean Piaget’s Birth,
Geneva, Switzerland, September.
Bereiter, C. & Scardamalia, M. (1985) Cognitive coping strategies and the problem of ‘inert
knowledge’, in: S. F. Chipman, J. W. Segal & R. Glaser (Eds) Thinking and learning skills,
vol. 2 (Hillside, N.J., LEA).
Bransford, J. D., Arbitman-Sith, R., Stein, B. S. & Vye, N. J. (1985) Improving thinking and
learning skills: an analysis of three approaches, in: J. W. Segal, S. F. Chipman & R. Glaser
(Eds) Thinking and learning skills, vol. 1 (Hillsdale, N.J., Lawrence Erlbaum Associates),
133–206.
Bukta, K. & Nikolov, M. (2002) Nyelvtanıtas es hasznos nyelvtudas [Language teaching and usable
language knowledge], in: B. Csapo (Ed.) Az iskolai m}uveltseg (Budapest, Osiris Kiado),
169–193.
Carroll, J. B. (1993) Human cognitive abilities (Cambridge, Cambridge University Press).
Csapo, B. (1990) Integrating the development of the operational abilities of thinking and the
transmission of knowledge, in: H. Mandl, E. De Corte, N. Bennett & H. F. Friedrich (Eds)
Learning and instruction. European research in an international context, vol. 2.2, Analysis of
complex skills and complex knowledge domains (Oxford, Pergamon Press), 85–94.
Csapo, B. (1992) Improving operational abilities in children, in: A. Demetriou, M. Shayer &
A. Efklides (Eds) Neo-Piagetian theories of cognitive development. Implications and applications for
education (London, Routledge), 144–159.
Csapo, B. (1996) Educating the mind: what school does and does not do, paper presented in the
symposium, ‘Shaping the mind through education’, ‘The Growing Mind’, International
Conference in Honour of the Centennial of Jean Piaget’s Birth, Geneva, Switzerland, September.
Csapo, B. (1997a) Development of inductive reasoning: cross-sectional measurements in an
educational context, International Journal of Behavioral Development, 20(4), 609–626.
Csapo, B. (1997b) Operational enrichment: improving operational reasoning through the content
of teaching, in: J. H. M. Hamers & M. Th. Overtoom (Eds) Teaching thinking in Europe.
Inventory of European programmes (Utrecht, Sardes), 235–239.
Csapo, B. (1997c) The validity of school knowledge, in: R. H. Lehmann, G. Venter, J. van Buer, S.
Seeber & R. Peck (Eds) Erweiterte Autonomie fur Schule. Bildungscontrolling and Evaluation
(Studien zur Wirtschafts- und Erwachsenenpadagogik aus der Humboldt-Universitat zu
Berlin), Band 13, 2, 129–140.
208 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
Csapo, B. (Ed.) (1998a) Az iskolai tudas [School knowledge] (Budapest, Osiris Kiado).
Csapo, B. (1998b) Az uj tudas kepz}odesenek eszkozei: az induktıv gondolkodas [Tools for creating
new knowledge: inductive reasoning], in: B. Csapo (Ed.) Az iskolai tudas (Budapest, Osiris
Kiado), 251–280.
Csapo, B. (1999) Improving thinking through the content of teaching, in: J. H. M. Hamers,
J. E. H. van Luit & B. Csapo (Eds) Teaching and learning thinking skills (Lisse, Swets &
Zeitlinger), 37–62.
Csapo, B. (2000) A tantargyakkal kapcsolatos attit}udok osszefuggesei [Students’ attitudes towards
school subjects], Magyar Pedagogia, 100(3), 343–366.
Csapo, B. (2001) Az induktıv gondolkodas fejl}odesenek elemzese orszagos reprezentatıv felmeres
alapjan [An analysis of the development of inductive reasoning on the basis of a large-scale
survey], Magyar Pedagogia, 101(3), 373–391.
Csapo, B. (Ed.). (2002) Az iskolai m}uveltseg [Literacy at school] (Budapest, Osiris Kiado).
Csapo, B. (2003) A kepessegek fejl}odese es iskolai fejlesztese [The development of competencies and their
improvement in a school context] (Budapest, Akademiai Kiado).
Csapo, B. (2004) Knowledge and competencies, in: J. Letschert (Ed.) The integrated person. How
curriculum development relates to new competencies (Enschede, CIDREE), 35–49.
Demetriou, A. (2004) Mind, intelligence, and development: a general cognitive, differential, and
developmental theory of the mind, in: A. Demetriou & A. Raftopoulos (Eds) Developmental
change: theories, models and measurement (Cambridge, Cambridge University Press), 21–73.
Demetriou, A., Shayer, M. & Efklides, A. (Eds) (1992) Neo-Piagetian theories of cognitive
development. Implications and applications for education (London, Routledge).
Dobi, J. (1998) Megtanult es megertett matematikatudas [Mathematics knowledge mastered and
understood], in: B. Csapo (Ed.) Az iskolai m}uveltseg (Budapest, Osiris Kiado), 169–190.
Glaser, R. (1984) Education and thinking. The role of knowledge, American Psychologist, 39(2),
93–104.
Habok, A. (2004) A tanulas tanulasa az ertelemgazdag tudas elsajatıtasa erdekeben [Learning to
learn in order to acquire meaningful knowledge], Magyar Pedagogia, 104(4), 443–470.
Hamers, J. H. M. & Csapo, B. (1999) Teaching thinking, in: J. H. M. Hamers, J. E. H. van Luit &
B. Csapo (Eds) Teaching and learning thinking skills (Lisse, Swets & Zeitlinger), 11–36.
Hautamaki, J., Arinen P., Eronen, S., Hautamaki, A., Kupiainen, S., Lindblom, B., Niemivirta, N.,
Pakaslahti L., Rantanen, P. & Scheinin, P. (2002) Assessing learning-to-learn: a framework
(Helsinki, Helsinki University).
Jozsa, K. (2002) Tanulasi motivacio es human m}uveltseg [Mastery motivation and literacy], in:
B. Csapo (Ed.) Az iskolai m}uveltseg (Budapest, Osiris Kiado), 239–268.
Karpati, A. (2002) A vizualis m}uveltseg [Visual literacy], in: B. Csapo (Ed.) Az iskolai m}uveltseg
(Budapest, Osiris Kiado), 91–133.
Korom, E. (1997a) Naiv elmeletek es tevkepzetek megjelenese a termeszettudomanyos fogalmak
tanulasa soran [Naive theories and misconceptions in learning science], Magyar Pedagogia,
97(1), 19–41.
Korom, E. (1997b) The effects of science education on children’s naive theories, paper presented at
the 7th European Conference for Research on Learning and Instruction, Athens, Greece, August.
Korom, E. (1998) Az iskolai tudas es a hetkoznapi tapasztalat ellentmondasai: termeszettudoma-
nyos tevkepzetek [Contradictions of school knowledge and everyday experience: science
misconceptions], in: B. Csapo (Ed.) Az iskolai tudas (Budapest, Osiris Kiado), 139–167.
Korom, E. (2000) A fogalmi valtas elmeletei [Theories of conceptual change], Magyar Pszichologiai
Szemle, 55(2–3), 179–205.
Korom E. (2005) Fogalmi feijl}odes es fogalmi valtas [Concept development and conceptual change]
(Budapest, M}uszaki Kiado).
Korom, E. & Csapo, B. (1996) Development of children’s misconceptions of the world, poster
presented at ‘The Growing Mind’, International Conference in Honour of the Centennial of Jean
Piaget’s Birth, Geneva, Switzerland, September.
Quality of learning outcomes 209
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
Molnar, E. K. (2000) A fogalmazasi kepesseg fejlodesenek merese [Measuring the development of
composition skills], Iskolakultura, 10(8), 49–59.
Molnar, E. K. (2002) Az ırasbeli szovegalkotas [Creation of written texts], in: B. Csapo (Ed.) Az
iskolai muveltseg (Budapest, Osiris Kiado), 193–216.
Molnar, Gy. (2001) Az eletszer}u feladat-helyzetekben torten}o problemamegoldas vizsgalata
[Examination of problem solving in real life situations], Magyar Pedagogia, 101(3), 347–373.
Molnar, Gy. (2006) Tudastranszfer es komplex problemamegoldas [Knowledge transfer and complex
problem solving] (Budapest, Muszaki Kiado).
Molnar, L. (2002) A kritikai gondolkodas [Critical thinking], in: B. Csapo (Ed.) Az iskolai tudas
(Budapest, Osiris Kiado), 217–237.
Nagy, L. (2006) Az analogias gondolkodas fejlesztese [Developing analogical reasoning] (Budapest,
M}uszaki Konyvkiado).
OECD (2003) Learners for life. Student approaches to learning. Results from PISA 2000 (Paris, OECD).
OECD (2004) Learning for tomorrow’s world. First results from PISA 2003 (Paris, OECD).
Szebenyi, P. & Vass, V. (2002) Tortenelmi tevkepzetek, tortenelemszemlelet, nemzeti azonossag-
tudat [History misconceptions and national identity], in: B. Csapo (Ed.) Az iskolai m}uveltseg
(Budapest, Osiris Kiado), 135–167.
Vidakovich, T. (1996) Development of deductive reasoning, poster presented at ‘The Growing
Mind’, International Conference in Honour of the Centennial of Jean Piaget’s Birth, Geneva,
Switzerland, September.
Vidakovich, T. (1998) Tudomanyos es hetkoznapi logika: a tanulok deduktıv gondolkodasa
[Scientific and everyday logic: students’ deductive reasoning], in: B. Csapo (Ed.) Az iskolai
tudas (Budapest, Osiris Kiado), 191–220.
Waeytens, K., Lens, W. & Vandenberghe, R. (2002) ‘Learning to learn’: teachers’ conceptions of
their supporting role, Learning and Instruction, 12, 305–322.
210 B. Csapo
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Tex
as]
at 1
7:40
30
Nov
embe
r 20
14
