Type of citizenship education

R

Review

A

edu

The EPPI-Centre

March 2005

systematic review of the

conducted by the Citizenship Education

impact of citizcation on student le

and achiev

is part of the Social Science Research Unit, Institute of Educatio

EVIEW

Review Group

enship arning ement

n, University of London

AUTHORS AND REVIEW GROUP Dr Ruth Deakin Crick University of Bristol Dr Marilyn Tew University of Bristol Dr Monica Taylor Journal of Moral Education/University of London Susan Ritchie University of East London Elspeth Samuel University of Bristol Keren Durant University of Bristol

CONTACT DETAILS

Dr Ruth Deakin Crick University of Bristol Graduate School of Education 35 Berkeley Square Clifton Bristol BS8 1JA

ADVISORY GROUP MEMBERSHIP Professor James Arthur Christchurch University College, Canterbury Ms Sheila Bloom Institute for Global Ethics, UK Trust Mr Tony Breslin Citizenship Foundation Dr Peter Brett St. Martin’s University College, Lancaster Dr Ian Davies Department of Educational Studies, University of

York Professor Jon Davison Institute of Education, University of London Mr Richard Davison Head of PSHE and Citizenship, St George

Community School, Bristol Dr Liam Gearon Centre for Research in Human Rights, University of

Surrey Mr Malcolm Lewis Graduate School of Education, University of Bristol Ms Janet Palmer Manchester Metropolitan University Mr Mark de Pulford Lord Chancellor’s Office Dr Sue Jackson Birkbeck College, University of London Mr David Kerr National Foundation for Educational Research Ms Sandie Llewellin The Post 16 Citizenship Group Professor Bart McGettrick University of Glasgow (Chair) Mr Don Rowe Citizenship Foundation Ms Jan Urban Smith Schools Advisor PSHE and Citizenship, Gloucester

CONFLICTS OF INTEREST A potential conflict of interest in this review is the fact that one of the authors of the review is also an author of one of the studies in it. The steps taken to minimise this risk were:

• The identification of papers through searching and keywording by the Review Group, with this paper being identified and keyworded by other members of the team

i

• A declaration of interest to the Review Group

• The double data extraction and moderation by other members of the Review Group

ACKNOWLEDGEMENTS This review was carried out with funding from the DfES through the EPPI-Centre. We would also like to acknowledge the support of the University of East London for the time made available for Susan Ritchie to work on this review. The help and advice of the EPPI-Centre staff who were linked to this review is gratefully acknowledged: Zoe Garrett and Rebecca Rees.

LIST OF ABBREVIATIONS AAIA Association for Achievement and Improvement

through Assessment AEA Europe Association for Educational Assessment AEI Australian Educational Index BeCal Belief, Learning and Information Gateway BEI British Educational Index CERG Citizenship Education Research Group CPD Continuing professional development DfES Department for Education and Skills DPE Deliberate psychological education EPPI-Centre Evidence for Policy and Practice Information and

Coordinating Centre ERIC Educational Resources Information Centre FHAO Facing History and Ourselves IC Instructional conversation ICT Information and communication technology IT Information technology ITT Initial teacher training LCPs Learner-centred principles NFER National Foundation for Educational Research OFTSTED Office for Standards in Education PSE Personal and social education PSHE Personal, social and health education QCA Qualifications and Curriculum Authority REEL Research Evidence in Education Library SCRE Scottish Council for Research in Education SCT Sentence Completion Test SoSig Social Science Information Gateway TAAS Texas Assessment of Academic Skills test TLI Texas Learning Index TTA Teacher Training Agency UK United Kingdom USA United States of America WoE Weight of evidence

ii

This report should be cited as: Deakin Crick R, Taylor M, Tew M, Samuel E, Durant K, Ritchie S (2005) A systematic review of the impact of citizenship education on student learning and achievement. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education.

The opinions expressed in this review are not necessarily those of the EPPI-Centre or the review funders. Responsibility for the views expressed remains solely with the review authors.

© Copyright

Authors of the systematic reviews on the EPPI-Centre website (http://eppi.ioe.ac.uk/) hold the copyright for the text of their reviews. The EPPI-Centre owns the copyright for all material on the website it has developed, including the contents of the databases, manuals, and keywording and data extraction systems. The centre and authors give permission for users of the site to display and print the contents of the site for their own non-commercial use, providing that the materials are not modified, copyright and other proprietary notices contained in the materials are retained, and the source of the material is cited clearly following the citation details provided. Otherwise users are not permitted to duplicate, reproduce, re-publish, distribute, or store material from this website without express written permission.

iii

http://eppi.ioe.ac.uk/

TABLE OF CONTENTS

SUMMARY .............................................................................................................1

Background.............................................................................................................1 Aims........................................................................................................................1 Review questions....................................................................................................1 Methods ..................................................................................................................2 Results....................................................................................................................3 Conclusions ............................................................................................................4 Implications.............................................................................................................5

1. BACKGROUND..................................................................................................7

1.1 Aims and rationale for current review ...............................................................7 1.2 Definitional and conceptual issues ...................................................................8 1.3 Policy and practice background......................................................................10 1.4 Research background.....................................................................................12 1.5 Authors, funders and other users of the review ..............................................19 1.6 Review questions............................................................................................19

2. METHODS USED IN THE REVIEW.................................................................22

2.1 User involvement ............................................................................................22 2.2 Identifying and describing studies...................................................................23 2.3 In-depth review ...............................................................................................28

3. IDENTIFYING AND DESCRIBING STUDIES: RESULTS................................31

3.1 Studies included from searching and screening .............................................31 3.2 Characteristics of the included studies ...........................................................33 3.3 Identifying and describing studies: quality- assurance results........................38

4. IN-DEPTH REVIEW: RESULTS .......................................................................39

4.1 Selecting studies for the in-depth review ........................................................39 4.2 Comparing studies selected for in-depth review with the total studies in systematic map.....................................................................................................39 4.3 Further details of studies included in the in-depth review...............................39 4.4.Synthesis of findings.......................................................................................40 4.5 In-depth review: quality-assurance results .....................................................58 4.6 Nature of actual involvement of users in the review and its impact ................58

5. FINDINGS AND IMPLICATIONS......................................................................59

5.1 Summary of principal findings.........................................................................59 5.2 Strengths and limitations of this systematic review ........................................62 5.3 Implications.....................................................................................................65

6. REFERENCES .................................................................................................68

6.1 Studies included in map and synthesis...........................................................68 6.2 Other references used in the text of the report ...............................................70

iv

Appendix 1.1: Advisory Group membership .........................................................74

Appendix 2.1: Inclusion and exclusion criteria......................................................75

Appendix 2.2: Search strategy for electronic databases ......................................77

Appendix 2.3: Journals handsearched .................................................................79

Appendix 2.4: EPPI-Centre keyword sheet, including review-specific keywords .81

Appendix 3.1: Details of studies included in the systematic map .........................84

Appendix 4.1: Comparison of studies in the systematic map and in-depth review..............................................................................................................................94

Appendix 4.2: Details of studies in the in-depth review ........................................96

v

Summary

SUMMARY

Background The context of this review is based in the English education system following the introduction of Citizenship Education as a statutory curriculum subject in 2002. The conceptual framework used for Citizenship Education therefore draws upon the Crick (1998) framework, which Crick defines citizenship education as including three distinct strands: moral and social responsibility, community involvement and political literacy. The first review in this series synthesised evidence from around the world of the impact of citizenship education on the provision of schooling; that is, on school leadership and management, learning and teaching, external relations and community; curriculum development, and school ethos and context. In addition, the impact on teacher professional learning was considered. The implications of this review indicated that citizenship education practices and processes were related to particular approaches to learning and teaching. This review builds on the findings of the first review to explore further the impact of citizenship education on student learning and achievement. These terms were used in an attempt to capture both the processes and the outcomes of learning that might be impacted on by citizenship education. Learning processes include those cognitive, affective, and volitional processes, activities and dispositions that operate in order for students to learn. Learning outcomes refers to achievement, when that is determined by a summative assessment either by teachers, or by tests or by examinations. Achievement refers to a level or standard of competence reached in a particular domain, which might be knowledge, skills or understanding in a subject of the curriculum, or in a particular behaviour or skill relating to personal or social development. It is achieved in relation to prior learning and attainment.

Aims The aims of this review are to explore the evidence available: • to determine the nature of the impact of citizenship education on student

learning and achievement

• to identify the implications of this for teacher education, both initial and ongoing

• to make recommendations for policy and practice based on these findings

• to identify further questions that need to be addressed by research

Review questions The main review question was as follows:

A systematic review of the impact of citizenship education on student learning and achievement 1

Summary

What is the impact of citizenship education on student learning and achievement?

This was refined at the second stage of in-depth review to the following:

What is the impact of citizenship education on students’ cognitive outcomes?

In order to achieve all the aims of the review, it was also considered necessary to address the following further question:

What are the implications of the findings of the review for teacher education?

Methods Studies for inclusion in the review were identified through electronic searches, journal handsearches, website searches and personal contacts. Inclusion and exclusion criteria were systematically applied to papers firstly on the basis of title and abstract, and secondly on the basis of the full text report. Review-specific and general keywords were applied to studies meeting the inclusion criteria and included in a systematic map of the research literature. The Review Group then decided to apply a second set of criteria at the in-depth review stage to focus further on studies that reported on cognitive outcomes.

In order to report findings relevant to the review questions, the studies were grouped in the synthesis according to their relevance to the following five categories of cognitive learning outcomes grouped under the theme of developing holistic approaches to achievement:

• meaning-making

• understanding and reasoning

• higher order thinking

• academic attainment

• communication skills

and five key learning processes:

• engagement

• promoting discussion

• learner-centred teaching

• meaningful curricula

• developing personally


Summary

Results The search strategy identified 647 reports as being potentially relevant. Application of the inclusion and exclusion criteria resulted in 35 studies being identified for inclusion in the systematic map, which included four studies from the first citizenship review. The studies included in the systematic map represented a range of study types, a wide range of citizenship education across different curriculum subjects, and the full range of learning processes and learning outcomes identified by the review-specific keywords. The application of second stage inclusion criteria resulted in 13 studies being included in the in-depth review, which addressed a range of types of citizenship education and a wide range of learning processes.

In summary, the findings of this review indicate that citizenship education can be applied to most areas of the curriculum through the development of learner-centred teaching, and meaningful curricula. The evidence from this review about the impact of citizenship education on student learning and achievement suggests the following:

Pedagogy appropriate for citizenship education:

• can enhance student learning and achievement

• may be characterised by a facilitative, conversational pedagogy, where dialogue and discussion are the norm

• can improve students’ communication skills

• can enhance students’ academic achievement

• can engage students to seek cognitive understanding of the meaning of their personal stories and experiences when learning about lesson content and gaining awareness of others’ situations

• may lead to greater participation when lesson content is pertinent to student experiences

• can enhance students’ higher order cognitive and intellectual development

• engages and can enhance students’ meta-cognitive processes

• allows for an increased participation and a greater interaction and may ensure a more positive experience of participation that affects student ability to make meaning of the lesson content

• can result in statistically significant positive changes in formal operations of movement from concrete literal thinking to abstract and scientific thinking, resulting in higher levels of reflection

• can create a cooperative learning environment, leading to an atmosphere of trust and safety, that enhances teacher/student relationships, where teachers let go of control and listen to student voice


Summary

• may empower students, leading to increased self-confidence, more positive self-concept and greater self-reliance

• can engage learners as whole persons and may result in teachers relating differently to students

• can impact on affective outcomes as well as cognitive growth in areas, such as the development of self-concept, increased self-confidence, and more positive behaviour

In addition:

• Teachers themselves may need support in order to develop expertise in facilitation and dialogue.

• Learner-centred teaching and meaningful curricula can affect the motivation and cognitive engagement of students and require a change from traditional teaching methods and content.

• Questioning and dialogue can encourage students in the processes of reflective searching for deeper meaning to issues and events.

• Cognitive outcomes are achieved in relation to the affective and volitional domain and not in isolation.

Conclusions

Strengths and limitations

A focus on cognitive learning outcomes might be considered to be a limitation, especially since citizenship education itself, and many studies of it, are often primarily concerned with personal and social learning (and sometimes moral and political learning). However, by deliberately placing cognitive learning in the spotlight, the review findings are able to show, by means of this clearer focus, that citizenship education pedagogies and curricular experiences can result in cognitive learning, as well as personal and social learning.

Most of the 13 research studies in the in-depth review were conducted in locations outside England and beyond the United Kingdom (UK). This is also the case even when the in-depth review studies are contextualised in the broader picture of the 35 research studies, which were keyworded. Furthermore, the two studies reviewed in-depth that were conducted in England were both undertaken prior to the implementation of the citizenship education curriculum in England in 2002. Moreover, given the small-scale nature of many of the studies and the numbers of schools, teachers and students involved, the generalisability of the specific findings may be limited. However, the interrelated cluster of overlapping categories and contributory processes which have emerged in the synthesis of findings, as a result of applying the EPPI-Centre review methods to the review question, gives the review’s evidence strength and pertinence.


Summary

A strength of the process was the independent double-checking of judgements of inclusion/exclusion criteria, keyword applications and data extraction by reviewers, and the further sample check by EPPI-Centre colleagues. On the other hand, the process can be seen as overly technicist, following a medical or economic model of data reporting, validity and reliability, whose presumed objectivity does not always fit so well with educational studies dealing with complex human interactions, learning processes and environments, and often subjective judgements about evidence of student learning.

Studies included in the review had not necessarily been undertaken to address issues close to the review question. The assessment of weight of evidence in relation to the review question enabled overall judgements to be made about the contribution of studies to the review. In this review, however, the papers were of varying quality; of those included in the in-depth analysis, a few studies were rated as having high trustworthiness and/or appropriateness or relevance, but only one was considered to have high review-specific weight of evidence and trustworthiness.

The foreshortened project deadline meant holding a meeting of the group at relatively short notice at the end of the traditional summer holiday period/ beginning of the school year, which members were unable to attend. Although written comments were invited, the timescale in effect reduced the common sharing in the consultative process. As a result, the Review Group was alone in being responsible for identifying the implications of the findings of this review for policy, practice and research.

Implications

If the findings of this review are taken seriously, they have clear implications for a radical review of the system and structure of schooling so as to incorporate citizenship education strategies to reconceptualise pedagogy as learner-centred and to develop approaches to achievement that are holistic.

Implications for policy

The findings of this review have particular implications for teacher professional learning. In initial teacher training (ITT) and continuous professional development (CPD), there is a need for teachers to be supported to develop a richer, deeper, broader process-oriented pedagogy. This involves having an understanding and vision of learning and achievement seen from a more holistic perspective, where different kinds or categories of learning are viewed as complementary, not separate.

Implications for practice

A citizenship pedagogy, based on the key themes characteristic of learning processes identified in the review, will have at its core communication, facilitating and enabling, dialogue and discussion, encouragement to engage with learning, and relating learning to experience. This more conversational and negotiated style of teaching and learning involves mutually respectful teacher-student relationships


Summary

where traditional authoritarian patterns of control are no longer appropriate. Citizenship education practices and processes that promote student learning and achievement cut across the curriculum and suggest the need for curricular flexibility, with more opportunities to develop different groupings of learners in interactive and conversational learning contexts.

Implications for research

There is a need for more interdisciplinary research, research which employs mixed methods, quantitative and qualitative, and which involves in-depth study of several schools. Studies are needed of the effects of different citizenship models and pedagogies on cognitive learning outcomes, and into ways to link such learning more carefully and systematically to complementary personal and social learning. Since citizenship education is about lifelong learning and practices, research should also investigate the interrelation between school-based learning in citizenship education and the family and community-based learning.


1. Background

1. BACKGROUND

This chapter introduces the concepts of citizenship education as they are used in this review, and discusses the concepts of learning and achievement and their relationship to concepts of citizenship education. It offers an educational rationale for the review question.

1.1 Aims and rationale for current review The first review undertaken by this group aimed to address the question of the impact of citizenship education on the processes and structures of schooling. The findings of the first review, based on 14 studies from around the world indicated that citizenship education does have a significant impact on school processes and structures.

The aims of this review are as follows:

• to determine the nature of the impact of citizenship education on student learning and achievement




It builds on the findings of the first review by aiming to explore the evidence available:

• to understand the links between citizenship education and the development of learning dispositions, values, attitudes and strategies

• to understand the relationship between citizenship education and higher order critical and creative thinking skills and the processes of learning

• to understand the relationship between citizenship education and learning outcomes

The purpose of this review was to address, through a synthesis of the evidence from research, the relationships between the teaching of citizenship education and student learning and achievement.

While ‘citizenship’ is a term that has entered educational discourse in England, through legislation, the ambit of citizenship education embraces much of what professional educators already hold to be important, although the language in which significance is expressed may differ. This Review Group was concerned that the research should not be narrowly focused on the current English experience of citizenship education, but should include an international dimension on policy, provision and practice, which would significantly strengthen the review


1. Background

through exploring generic and specific issues relating to citizenship education from other cultures.

1.2 Definitional and conceptual issues

1.2.1 Citizenship education

Citizenship education is understood as all of those planned experiences that school-based educators construct for their students in order to fulfil the different aims and purposes of citizenship education. These may be formal or informal, explicit or implicit, extra-curricular, cross-curricular or within particular curriculum strands. Citizenship education also includes the provision for pastoral and personal development of students and thus relates to both pedagogy and school ethos and culture.

The conceptual framework that has been adopted for this review is drawn from Crick (1998). This framework has been selected because it was itself the outcome of considerable research, development and consultation, drawing on a wide range of processes that together were referred to at the time, as forming ‘preparation for adult life’ initiatives. It is also a framework that defines the scope of citizenship education in England and although its terminology is contested, it is ‘maximal’ in its scope and provides a broad framework around which to focus the study.

The key elements of the Crick framework are outline below.

Moral and social development

The report describes this aspect of education as near the heart of citizenship education and it includes the development of moral values and personal development, which takes place for students in a wide range of contexts and in many different ways.

Community involvement

This is described as learning about and becoming helpfully involved in the life of the community, and in service to the community. The community includes firstly the school, and then the wider local and national communities of which students are a part.

Political literacy

Political literacy encompasses learning about becoming effective in public and community life through knowledge, skills, values and attitudes, which support active engagement. In addition to this, the National Curriculum sets out what schools are required to teach under these headings, and the expectation is that students will:

• have knowledge and understanding about becoming informed citizens

• develop skills of enquiry and communication

• develop skills of participation and responsible action


1. Background

1.2.2 Learning and achievement

The current review seeks to examine the impact of citizenship education on learning and achievement. These terms are used in an attempt to capture both the processes and the outcomes of learning that may, theoretically, be impacted by citizenship education. It is considered that the review question requires a focus on both the processes of learning as well as the outcomes and will therefore necessarily appear to favour pupil-centred learning and a holistic approach to education. The review does not exclude subject-based, or didactic approaches to learning and achievement, which may often be appropriate to teaching about citizenship, as in the Political Literacy strand of the National Curriculum.

Learning processes

Learning processes include those cognitive, affective and volitional processes, activities and dispositions that operate in order for students to learn. The findings of the first review had implications for the processes of learning. Findings of particular relevance are:

• the need for contextual knowledge and problem-based thinking for citizenship

• learning about shared values, human rights, and issues of justice and equity

• taking part in educational experiences that are challenging, attainable and relevant to students’ lives and narratives

• experiencing a quality of dialogue and discourse that provides opportunities to engage with values issues and empowers them to voice their view, and name and make meaning from life experiences

• inclusive and respectful pedagogy for learners and teachers

Learning outcomes

‘Learning outcomes’ refers to achievement, when that is determined by a summative assessment either by teachers, or by tests or examinations. The domains of achievement are not limited to cognitive outcomes, although these are the most readily understood, as, for example, in the knowledge, skills or understanding required for particular levels in a subject of the curriculum. Specific curricular outcomes, such as these, are often referred to as ‘attainment’. Achievement outcomes may also be evidenced by particular behaviours or skills relating to personal or social development, including affective and interpersonal domains. For the purpose of this review, ‘achievement’ is understood as a learning outcome, which may be assessed by teachers or students and which may pertain to cognitive, personal, social, emotional, or moral/political domains of human experience.

The term ‘attainment’ is used to identify the exact standard or judgement of what a student knows, understands or can do, in relation to curricular goals. The most common attainment measures in schools are summative assessment measures or tests, graded by externally provided criteria and often norm-referenced against the population. Achievement, however, includes a ‘value added’ dimension, in relation to where the student started from in their learning, and can include learning outcomes of different kinds.


1. Background

Achievement in citizenship education, for example, could be determined by a portfolio or narrative account of particular service activities or experiences a student has recorded and evaluated. Less common, achievement measures could be a ‘citizenship award’ for particular service to the community, or the ‘learner of the week’ award in a classroom where the processes of learning are valued as much as the outcomes. It could also be a teacher or self-assessment of a student’s social skills, such as empathy or conflict resolution.

Learning outcomes are assessed in relation to the processes of learning that have been taking place. Some teacher assessments or tests may assess evidence of higher order thinking skills, and these would be described as ‘cognitive learning’ outcomes. Other teacher assessments may judge evidence of student activity or engagement in the community; these would be described as ‘social learning outcomes’. It is also possible, though less common, for personal outcomes to be assessed through self-report measures or teacher observation, and these may also include moral and political outcomes.

1.3 Policy and practice background

1.3.1 Citizenship Education

Citizenship education became a statutory requirement for secondary schools in England from September 2002 and a recommended subject for primary schools with guidelines in the form of a framework for personal, social and health education and citizenship. The history of the development of this policy initiative spans the 1990s and has been informed by the requirements of the 1992 Education Act, which required the Office for Standards in Education (Ofsted) to report on the spiritual, moral, social and cultural development of students in schools. Significant among the resulting policies were developments in personal, social and health education; and a range of initiatives, which addressed the personal and social aspects of student development, including the National Forum for Values in Education and the Community. All these initiatives have been informed by growing societal concerns about values and the personal development of young people. Resources for citizenship education are provided from within the voluntary sector and from government departments.

Alongside these policy developments, there has been a much greater emphasis on improving standards in education, focusing mostly on measurable learning outcomes but, more recently in the UK, a greater emphasis on the processes of learning, and the use of assessment for learning rather than simply of learning outcomes. These policy initiatives are generally seen as distinct and separate from the initiatives surrounding citizenship and values. However, as has been identified in the first of these systematic reviews (http://eppi.ioe.ac.uk/reel/), when schools address citizenship education, defined in its broadest sense, then there are implications for the core tasks of schooling and in particular for the context and manner in which students learn.

Within England, and to some extent the whole of the UK, the Crick Report (1998) provides the current framework for citizenship education. Crick (1998) defines citizenship education as including three distinct strands: moral and social responsibility, community involvement and political literacy. The programmes of


http://eppi.ioe.ac.uk/reel/

1. Background

study for the National Curriculum appear to focus more on political literacy, but many of the outcomes are in the domain of personal development. Citizenship is linked in these documents to whole school ethos and organisation, to values education and to the spiritual, moral, social and cultural development of students. While much discretion is left to individual schools, it is clearly expected that citizenship education will appear in discrete curriculum time, across the whole curriculum and in extra-curricular activities and be related to the school’s particular vision and values.

The report identifies four distinct elements of Citizenship Education, which it suggests should be reached by the end of compulsory schooling. These are key concepts, values and dispositions, skills and aptitudes, and knowledge and understanding. The skills in particular relate to cognitive and social learning processes, while the values and dispositions relate to moral concerns some of which are reminiscent of Smith and Spurling’s (1999) moral components of lifelong learning. The skills are as follows:

• ability to make a reasoned argument, both orally and in writing

• ability to co-operate and work effectively with others

• ability to consider and appreciate the experience and perspective of others

• ability to tolerate other viewpoints

• ability to adopt a problem-solving approach

• ability to use modern media and technology critically to gather information

• a critical approach to evidence put before one and an ability to look for fresh evidence

• ability to recognise forms of manipulation and persuasion

• ability to identify, respond to and influence social, moral and political challenges and situations

In a review of how teacher educators were approaching citizenship, McGettrick (2002) identified three broad approaches. These are as follows:

• A process approach, which focuses mainly on the context of schooling and the ways in which schooling is organised. It is developmental in style, unpredictable in intent, varied in methodologies and flexible in programme design.

• A transmission approach, which focuses on the knowledge content of citizenship methodologies, and is inflexible in programme design.

• A transformational professional approach, which is concerned with the re-professionalisation of teachers as facilitators of learning and focused on learning as a process for transforming society. It is based on the teacher as a primary agent of social change, developed in a community context, adaptable to widely changing circumstances, and flexible in programme design.


1. Background

These three broad approaches are indicative of the diversity with which citizenship education is currently being addressed by teacher educators and by schools themselves, and the breadth of learning processes and outcomes addressed by Citizenship Education.

1.4 Research background Citizenship Education and lifelong learning

Citizenship Education1, as it is now addressed in England, is a relatively new subject, although some of its content, such as moral education, has been present in education policies worldwide for many years. There is a growing body of research into Citizenship Education that is beginning to survey the field. For example, Kerr (1999a; 1999b; 2002; 2003) reveals a number of common sets of issues and challenges which are facing education systems worldwide and which have led to the contemporary focus on citizenship education and its related themes. Although these themes are often disparate, they have in common a focus on schooling outcomes other than traditional summative achievement outcomes. Thus the domain of citizenship education includes all those planned interventions in schooling that have as their purpose a personal or social outcome, not just a focus on academic or vocational qualifications. It is the overlap between the personal and social development of students and their development as effective learners that forms the focus of the question for this review.

Smith and Spurling (1999) developed a holistic notion of lifelong learning that includes an empirical element that describes the scope of lifelong learning and a moral element that reflects four principles of conduct. They suggest that the emphasis should be on continuity, intention and unfolding strategy in personal learning, and that linked to these are four principles of personal commitment to learning, social commitment to learning, respect for others’ learning and respect for truth.

Smith and Spurling further argue that there are four discourses that support this definition of lifelong learning, which are linked conceptually with the values of democratic debate. These have to do with the value of group learning (Clark, 1997), in which the key dynamic is that individual members feel a sense of common purpose in the group, while at the same time they feel that the group recognises and values their individual contribution and potential. They argue that personal and social commitment to learning produces ‘public goods’ which are vital to everyday social and economic life (Gray, 1999) and they suggest that, underlying this view of lifelong learning, is a theme, which is consistent with a widespread ethical concern for equality of opportunity in lifelong learning. Finally, they indicate that the moral principles of lifelong learning, especially those of respecting others’ learning and respecting truth, affirm Friere’s (1972) idea of learning as a ‘naming of the world’.

1 The term ‘Citizenship Education’ is used when referring to a planned curriculum, such as that within the English National Curriculum. This is sometimes also called ‘explicit Citizenship Education’. When generic citizenship education is referred to, sometimes also called ‘implicit citizenship education’, then lower case will be used.


1. Background

Thus, in mapping out an overall picture of what lifelong learning involves, Smith and Spurling begin to develop ideas about learning identity, or the active learning agent within community. This concept of learning identity is important because it underlies the notion of a range of learning capabilities.

A broad view of lifelong learning is consistent with Bloomer’s (2001) analysis of learning, based on a longitudinal study of young people and their experiences of learning. He concludes that the explanatory power of learning theory is enhanced when it includes a temporal dimension – to do with learning over time, and when it addresses how learning is embedded within the complexity of life experiences.

In 1990, the American Psychological Association appointed a task force for Psychology in Education. The purpose of this task force was to review over a century of research on learning, motivation, development and individual differences. It led to the development and dissemination of the learner-centred psychological principles (LCPs). In 1997, the task force originally identified 12 principles, which were revised as 14 statements (http://www.apa.org/ed/lcp.html). According to the preamble:

These principles emphasize the active and reflective nature of learning and learners. From this perspective, educational practice will be most likely to improve when the educational system is redesigned with the primary focus on the learner.

These principles identified learning as a whole person phenomenon (McCombs, 1997). The principles include cognitive and meta-cognitive as well as motivational and affective, social and developmental, and other individual difference factors important to optimal learning, motivation and development. Practices based on these principles are consonant with recent discoveries from psychology relating to positive youth development and prevention interventions (Seligman and Csikszentmihalyi, 2000). Thus it seems that there is evidence of links between learner-centred approaches to education and many of the concerns of citizenship education.

Learning/Learning processes

In order to explore the possible links between citizenship education and the process of learning, it is necessary to examine some of the research evidence already available about learning processes, in order to map the terrain for this review. Johnston (1996) proposes a model of learning that includes three domains of affect, cognition and conation, which have a learning focus of ‘I feel’, ‘I know’ and ‘I act’ (see Figure 1.1). This provides a helpful map of the types of domains that might be included in an understanding of learning processes. In addition, creative and critical thinking skills, learning dispositions, and inter- and intra-personal awareness are all likely to be relevant.

Discussion of the development of thinking and of learning to learn has been characterised by a plethora of terminology used to describe the kind of thinking under discussion, so much so that Cuban (1984) described the area as ‘a conceptual swamp’ (p 676). As the current review seeks to understand the relationship between citizenship education and the processes of learning, there is a need for clarification of terms. It is important, for instance, to explore and determine the similarities and differences between: higher order thinking skills;


http://www.apa.org/ed/lcp.html

1. Background

problem-solving; critical thinking; reasoning; decision-making; meta-cognition; productive thinking; creative thinking; and mindful thinking.

This task is rendered even more complex by changes to the meaning of these terms over time and according to the academic disciplinary context (for example, philosophy or psychology). For example, Lewis and Smith’s (1993) discussion of the definition of higher order thinking noted that critical thinking has in the past been regarded as synonymous with problem-solving, while later it was regarded as evaluation or judgement. Yet again, at other times it has been treated as a combination of evaluation and problem-solving.

Figure 1.1: The interactive learning model (Johnston, 1996)

Lewis and Smith’s position is that the term ‘higher order thinking’ encompasses problem-solving, critical thinking, creative thinking and decision-making. They define it as follows:

Higher order thinking occurs when a person takes new information and information stored in memory and interrelates and/or rearranges and extends this information to achieve a purpose or find possible answers in perplexing situations. (Lewis and Smith, 1993, p 136)

In relation to the kind of learning implicit in an exploration of values, human rights, and issues of justice and equity, this is perhaps a more helpful definition. However, Newman (1990) pointed out that what may require higher order thinking by some may only require lower order thinking by others and so ‘to determine the extent to which an individual is involved in higher order thinking, one would presumably need to know something about the person’s intellectual history’ (Newman, 1990, p 45).


1. Background

A useful model in relation to citizenship education, particularly as it relates to the findings of the first review, is that proposed by Fogarty and McTighe (1993). They identified three phases in the attempt to define and teach thinking skills. These are:

• skill acquisition (e.g. Marzano and Hutchins, 1985)

• critical and creative thinking as required for problem-solving (e.g. de Bono, 1983)

• ‘thoughtful application’ (Brown and Palinscar, 1982)

Clearly, skill acquisition and critical and creative thinking are important for decision-making and lifelong learning, as required in a rapidly changing world. Yet it is the third kind of thinking skill, that of ‘thoughtful application’, that seems to be the most relevant to citizenship education as described in the findings of the first review. Fogarty (1992) calls this kind of thinking ‘the thoughtful classroom / the mindful school’. It is the level of thinking required to shift from a direct instruction model that focuses on skill development to a level of thinking where learners actively process the information in order to construct knowledge and meaning. This third level or phase of thinking requires thoughtful abstraction of information for application and transfer of learning. Fogarty and McTighe (1993) argues that the thoughtful classroom and school is one where students develop ‘productive problem-solving strategies, mindful decision-making tactics and creative, innovative thinking’ (Fogarty and McTighe, 1993, p 165).

Fogarty (1992) argues that there has been an evolutionary path in the thinking skills movement from skill acquisition, to meaning-making and finally to application and transfer. He called this the ‘three story intellect’: the first story is where teachers are concerned with teaching the specific skills of thinking, such as creative skills and critical skills; the second story is where teachers are concerned with providing opportunities for students to practise those skills with appropriate tools, such as co-operative learning structures and graphic representations; and the third story is where teachers help students to anchor their learning, using the processing methods structured into second story skills and they build up concepts, skills, attitudes and strategies for lifelong use, and application in diverse academic, social and personal settings throughout their lives. He suggests there is an overlap between creative and critical thinking, which draws on the affective domain and leads to the application and transfer of knowledge. These he suggests are significant curricular outcomes for the thoughtful classroom and the mindful school. They are identified in Figure 1.2 and show a relationship between creative and critical thinking, and affective development.


1. Background

Figure 1.2: Fogarty and McTighe (1993)

AffectiveSense of humour

Efficacy as a thinkerCooperativeCollaborativeRisk taking

Perseverance

Cognitive CognitiveBothMetacognitive

ReflectionUse of past knowledge

Communication skills

TransferAccessing informationProblem

solvingDecision making

Creative thinking(Generative productive)

Flexible thinkingFluent thinkingWonderment

InquisitivenessCuriosityIngenuityOriginality

InsightfulnessElaborative thinking

Critical thinking(Analytical evaluative)

PrecisionAccuracy

Ability to critique reference criteria

PrioritisationTolerance for ambiguity

Organising

Another writer who has influenced the definitions of creative and critical thinking is Adey. He frames thinking as a layered structure (Adey, 2001). One layer is the general processing ability, variously described as intelligence, working memory, formal operations, multivariate thinking and higher order thinking skills. In a different layer, there are particular thinking skills, of which critical and creative thinking, were seen by Adey as ‘key nodes’. Adey identified critical thinking as analytical thinking, a convergent type of thinking, related to reasoning and to formal and informal logic. Creative thinking he describes as divergent and lateral; the type of thinking that generates new ideas. Other skills that can be regarded as derivatives of these skills are evaluative thinking, exploratory thinking and hypothesising.

Adey, like Fogerty and McTighe, sees problem-solving as a combination of all other thinking skills, calling on some more than others according to the demands of the problem. This means that problem-solving is not readily defined, since it will in some circumstances draw more on convergent critical thinking and, in others, on divergent creative thinking.

Ennis (1993, 1996) also defines a critical thinker in terms of convergent skills, such as the ability to judge the credibility of sources, the quality of argument, and to identify conclusions, reasons and assumptions; to develop and defend a position on an issue; to plan experiments and to judge experimental design.

Huitt (1998) agrees with this in offering the definition of critical thinking as ‘the disciplined mental activity of evaluating arguments or propositions and making judgements that can guide the development of beliefs and taking action’ (p 2). He also gives a useful definition of non-critical thinking as thinking which does not consider current data but relies on past practices, or brainstorming that involves saying whatever comes to mind without evaluation. Non-critical thinking also includes creative thinking, which involves synthesis and requires ‘an individual to


1. Background

look at parts and relationships (analysis) and then to put these together in a new and original way’ (Huitt, 1998, p 2).

Lawson approaches thinking as a series of ‘skills’ or ‘the ability to do something well’ (Lawson, 1993, p 171). Others have dropped the word ‘skill’ in favour of referring to ‘thinking’.

In the context of the current review, where the concern is with the impact of citizenship education on learning and achievement, a focus on ‘thinking’ seems to be more appropriate than a list of skills. Thus, although it would not be inappropriate, it would not be essential to use the term ‘skill’. What is more important is to be clear that we are concerned with outcomes of the kind listed by Fogarty, as these are closely aligned with what is required by students for dealing with choices, problems and challenges in everyday life and for continuing to learn from experience and from their own thinking.

Thus it seems that a particular link between citizenship education and the processes of learning can be identified in the development of higher order thinking skills. These are already identified in the National Curriculum framework in England along with particular values, dispositions, aptitudes and knowledge.

Educational issues

In order to adopt any approach to citizenship education, at school or initial teacher training level, it is critical to have an informed view of the scope of education for citizenship in relation to content, pedagogy, leadership and management, ethos and external relations. Views of politicians and lobbyists are not sufficient; educators necessarily respond from a different perspective, which is that of the informed practitioner whose central task has to do with learning within a particular community. There has been very little serious research that has addressed models of practice for citizenship education at a school-based level (Deakin Crick et al., 2004), and very little attempt to integrate notions of citizenship education into broader educational philosophies and practices.

From an educational perspective, there are a number of crucial issues that need to be addressed in relation to citizenship education. The first review undertaken by this Review Group aimed to address the question of the impact of citizenship education on the processes and structures of schooling (Deakin Crick et al., 2004). The findings of that review, based on 14 studies from around the world, indicate that citizenship education does have a significant impact on school processes and structures. The implications of this are important for school leaders as they address policies relating to teaching and learning; leadership and management; school ethos and context; external relations; and community and curriculum construction and development.

The combined findings relating to these themes are summarised here:

• The quality of dialogue and discourse is central to learning in citizenship education.

• Dialogue and discourse are connected with learning about shared values, human rights, and issues of justice and equity.


1. Background

• A facilitative, conversational pedagogy may challenge existing power structures.

• Transformative, dialogical and participatory pedagogies complement and sustain achievement rather than divert attention from it.

• Such pedagogies require quality of teacher-student relationships that are inclusive and respectful.

• Students should be empowered to voice their views, and to name and make meaning from their life experiences.

• Contextual knowledge and problem-based thinking can lead to (citizenship) engagement and action.

• Engagement of students in citizenship education requires educational experiences that are challenging, attainable and relevant to students’ lives and narratives.

• Opportunities should be made for students to engage with values issues embedded in all curriculum subjects and experiences.

• A coherent whole-school strategy, including a community-owned values framework, is a key part of leadership for citizenship education.

• Participative and democratic processes in school leadership require particular attitudes and skills on the part of teachers and students.

• Listening to the voice of the student leads to positive relationships, an atmosphere of trust and increased participation. It may require many teachers to ‘let go of control’.

• Teachers require support to develop appropriate professional skills to engage in discourse and dialogue, and to facilitate citizenship education.

• Strategies for consensual change have to be identified by, and developed in, educational leaders.

• Schools often restrict participation by students in shaping institutional practices but expect them to adhere to policies and this can be counterproductive to the core messages of citizenship education.

Implications of the findings of the first review

The implications of the findings of the first review for policy-makers focus around the need for ongoing teacher education and professional learning, local decision-making and diversity, and the need to support teachers in making their own professional judgements in the classroom and school practice. The implications for practice suggest that citizenship education is an integral part of the core task of schooling.

The findings suggest that in classrooms the quality of dialogue and discourse is central to learning in citizenship. Such dialogue requires that students are empowered to voice their views, to name and make meaning from their life


1. Background

experiences; this quality of dialogue is supported by a quality of teacher-student relationship that is inclusive and respectful. Students are encouraged to make or create their own meaning from information and experiences through a process of reflection and dialogue. Further, engagement of students in citizenship education requires educational experiences that are challenging, attainable and relevant to students’ lives and narratives. Implicit in these findings is also a focus on higher order critical and creative thinking skills and the processes of learning itself, a point explored in greater detail in section 1.2 (Definitional and conceptual issues).

Amongst the implications of these findings is the need for curriculum and assessment practices that are learner-centred, which honour the student voice; develop positive interpersonal relationships; stimulate higher order thinking; and cater for individual differences. In research terms, the review suggested a significant gap in the body of available empirical or theoretical research that addresses the implementation of citizenship education at a school level and that there is a need for greater rigour and awareness relating to matters of quality and value.

1.5 Authors, funders and other users of the review The review question has emerged from a perceived need which has been made more urgent by the statutory requirement for higher education institutions to train teachers as teachers of Citizenship Education, alongside students who are training to be teachers of traditional subjects (Teacher Training Agency (TTA), 2004). One of the contributors to the review is the CitizEd project, a network of all higher education providers of ITT in citizenship education in England and funded by the TTA (http://www.citized.info/). The experience of this group is that there is a significant gap in professional understanding of what citizenship education actually is; what the subject knowledge might be; what the links with learning and schooling generally are; how citizenship education is best implemented in practice; and its implications for school leadership and management. Such a gap in knowledge requires the development of a substantial research base and dissemination network. The Citizenship Education Research Group (CERG) was set up in response to this need. The members of the steering committee of the CitizEd Project are also members of the CERG, which also includes a strong representation of policy-makers and non-government organisations which are providers of resources for citizenship education, as well as the newly formed Professional Association of Citizenship Teachers. The reviewers include researchers and teacher educators, who between them have significant experience of school and wider educational leadership and management.

1.6 Review questions The purpose of this review is to address, through a synthesis of the evidence from research, the relationships between the teaching of citizenship education and student learning and achievement.

In summary, the aims of the study are as follows:


http://www.citized.info/

1. Background

• to determine the nature of the impact of citizenship education on student learning and achievement




Scope of the review

The overall question to be addressed in the review is as follows:


In order to achieve all the aims of the review, it will be necessary to address the following further question:


The term ‘impact’ is used in this review question with care. It could imply a linear, cause-effect relationship between an independent variable and a dependent variable. However, the Review Group was concerned that such a narrow definition might not do justice to the complexity of the two variables, and the iterative relationship between them. However, the review focuses on the nature of the relationship between the two variables and impact in the direction of the impact of citizenship education on learning and achievement. There were no studies found which showed the impact in the other direction.

Addressing these questions involved seeking out a range of studies of different types. Both intervention studies and non-intervention studies are relevant. Intervention studies are those in which a specific citizenship intervention programme was introduced in order to study the effect of citizenship education on student learning and achievement. These studies could be either researcher manipulated (e.g. the researcher had introduced the intervention), or naturally occurring (e.g. the citizenship programme was already occurring prior to any evaluation). These interventions took the form of quasi-experimental studies and some involved the comparison of results between groups who received the intervention and those groups who did not. Non-intervention studies did not involve intervention programmes but included surveys of existing conditions relating to citizenship education in order to describe and identify associations between citizenship programmes and learning processes and outcomes.

The review was concerned with citizenship education during the period of schooling covering the age range 4–18 years. The review has included evidence from work relating to citizenship in all areas of the curriculum, as well as work contained within discrete ‘Citizenship Education’ curriculum time and extra curricular activities including community participation.

The outcomes of this work are of four kinds:


1. Background

• Through a process of consultation with users, the implications for citizenship education will be identified at national and local levels, and for the practices of teachers and teacher educators.

• A set of reports, tailored in style and length to different user groups, will be produced and disseminated.

• Electronic publication of the reports and databases produced by the review will appear in the Research Evidence and Education Library (REEL) maintained by the EPPI-Centre. Dissemination through REEL will be tailored to different end-users and potential audiences.

• The kinds of further research, both in terms of research questions and of methods, that are needed in this field will be identified.


2. Methods used in the review

2. METHODS USED IN THE REVIEW

This chapter gives details of the systematic methodology used for the review. It describes the role of the Review Group, and user involvement in the review process. It describes in detail the search methodologies, the location of the screening process, the keywording methods, the process of data extraction, and the quality-assurance process. It gives an overview of the in-depth review, the data-extraction process, and the synthesis of findings.

2.1 User involvement The Review Group was set up to represent a range of expertise in the field of citizenship education, with a particular emphasis on academics and practitioners who are engaged in the process of implementation of citizenship education in schools and in the education of teachers for citizenship. The Review Group works collaboratively with the professional resource network for initial teacher training in Citizenship Education, known as CitizED (http://www.citized.info/), a project of the Teacher Training Agency based at Christchurch University College, Canterbury.

2.1.1 Approach and rationale

Citizenship education is a complex concept, the definition of which remains problematic. While there is a growing body of research into aspects of citizenship education (Kerr et al., 2001; Torney-Purta et al., 2001; Kerr, 2002; Gearon, 2003) there is little that explores the implementation of citizenship education in schools in relation to the core purposes and processes of education (Deakin Crick et al., 2004). This review, like the first one, specifically explores the impact of citizenship education on schooling, and, as such, it is important to consult practitioners and those who apply research in practice. This has been the intention of the Review Group.

2.1.2 Methods used

The Citizenship Education Research Strategy Group was formally registered in November 2002 and the first formal meeting of the group was held on 7th November 2002. The group received detailed reports of the first review, and contributed to the review process at key points. As a matter of principle, the group determined that the following issues should be borne in mind through the process of the reviews:

• The breadth of the field of citizenship education, which is understood by the Review Group to cover the widest possible range of activities that schools undertake under the general heading of ‘preparation for adult life’ and ‘personal development’

• The need for comparative studies and awareness of the differing terminology used for the field in different countries




• The different stakeholders and interest groups involved in citizenship education

• The need for this Review Group to be looking at the research question from the point of view of school-based educators implementing citizenship education, and the possible differences between this and government or charitable perspectives of what citizenship education should be

This review question has been the focus of consultation since September 2003 and the final formulation is stated in this report. The members of the Citizenship Education Research Strategy Group are listed in Appendix 1.1, and include teachers, advisors and teacher educators. Meetings of the CERSG were arranged to coincide with the main decision points:

• reviewing the map of research and identifying exclusion and inclusion criteria for selecting studies for in-depth review, May 2004

• discussing the synthesis of findings, July 2004

• drawing up recommendations, July 2004

• reviewing the draft report, September 2004

2.2 Identifying and describing studies The review question required a search strategy that identified studies reporting on empirical research and that addressed the relationship between a citizenship intervention and student learning or achievement. As has already been made clear from the first review, the concept of citizenship education is complex. Furthermore, there is no consensus about the definition of learning or achievement in relation to citizenship education. Thus the task of searching was difficult and required rigorous criteria. Search terms were utilised for citizenship, learning, achievement, and schools for the electronic databases of ERIC 84-89 and ERIC 90-04; these are in Appendix 2.2.

2.2.1 Defining relevant studies: inclusion and exclusion criteria

Studies were included if they met all the inclusion criteria.

(i) Type of citizenship education

The study reports, as a specific focus, on an aspect of citizenship education, that is a planned curricular intervention by schools that directly aims to address education for citizenship. This could be delivered as a discrete part of a school timetable or it could be cross-curricular or extra-curricular. As noted in the background to this review, citizenship education includes a range of important aspects of schooling, including personal development and spiritual, moral, social and cultural development, political literacy, community involvement and other terms in use, such as service learning or character education.



(ii) Type of outcome

The study reports on the process of learning and/or student achievement. For the purpose of this review, the former consists of learning dispositions, higher- order thinking skills, meta-cognition, inter- and intra-personal awareness of learning, collaborative learning and motivation for learning. The latter consists of academic, personal or social achievement as assessed by teachers, examinations or self-assessment by students.

(iii) Type of study and study design

The study is empirical. It might be a descriptive study that looked at any aspect of citizenship education (descriptive study) as implemented in the context of student learning or achievement; or a study which explores the relationship between an aspect of citizenship education and student learning and achievement; or a study that was an evaluation of a naturally occurring or researcher-manipulated intervention, which was a citizenship education intervention (evaluation) which addressed student learning or achievement. Reviews and meta-analyses were not considered appropriate for data extraction but formed a useful background reference for the review. A study that describes or theorises about an aspect of citizenship education and its relationship to learning and achievement, without being based directly on empirical research, was not included. An empirical study was defined as one that reports on research and provides information about a specific research question, data collection and data analysis processes, and the findings that emerge from these.

(iv) Setting and population

The study reports on students in school or pre-school, between the ages of 4 and 18. Studies dealing only with students outside this age range, in further or higher education or adult education, were excluded.

(v) Date of research

Since the concern of the review related to the impact of citizenship education, relevant studies were considered to be those reported in 1988 or after – that is, since the beginning of the period of major policy changes with regard to citizenship and preparation for adult life in the UK, and many other countries.

(vi) Language of report

Studies were included only if they were written in English.

The following labels were attached to each excluded record to identify the reason for exclusion:

a1 not citizenship education a2 not learning a3 not achievement b not schools c not conducted post 1988 d not English language e not empirical research



Studies that were considered relevant for background reading for the review were excluded and labelled BR.

2.2.2 Identification of potential studies: search strategy

Reports were identified in the following ways/from the following sources:

(i) Bibliographic databases: Educational Resources Information Centre (ERIC), British Educational Index (BEI), Australian Educational Index (AEI), Psychological Literature (PsycInfo)

(ii) Web-based information gateways: Social Science Information Gateway (SoSig), Belief, Learning and Information Gateway (BeCal)

(iii) Specialist registers: research registers of the National Foundation for Educational Research (NFER), Scottish Council for Research in Education (SCRE)

(iv) Search of journal publisher’s web pages and handsearching of key journals (see later list)

(v) Personal contacts

(vi) Scanning the reference lists of already identified reports

(vii) Direct requests to educational research institutions and association members: Association for Educational Assessment (AEA Europe), Association for Achievement and Improvement through Assessment (AAIA), NFER, Qualifications and Curriculum Council (QCA)

Search terms for searching bibliographic databases included the following sets in combination (details of the terms used are presented in Appendix 2.2):

• Terms to indicate that a study was about citizenship education

• Terms to indicate that a study involved school students aged from 4 to 18

• Terms to indicate that a study concerned an aspect of learning or achievement

ERIC search terms were used as a start for the search strategy. For each of the search terms in Appendix 2.2, related terms in the index were selected as they applied to the research question of the review. For instance, if the search term ‘moral education’ is entered in an advanced ERIC search, the further terms are also provided, as shown in Table 2.1.



Table 2.1: ERIC search terms Number of items Related terms Search terms 61 No related terms Moral education 0 No related terms Moral instruction 504 5 related terms Moral issues 1 2 related terms Moral judgement 1 No related terms Moral judgement task (Piaget) 1 No related terms Moral judgement test 1 No related terms Moral leadership 1 No related terms Moral majority 18 No related terms Moral orientation 6 No related terms Moral reality 1 No related terms Moral rearmament 160 No related terms Moral reasoning 1 No related terms Moral reflection 1 No related terms Moral relativism 1 No related terms Moral theory 3 No related terms Moral transgression 2355 No related terms Moral values 1 No related terms Moral vocationalism 1038 16 related terms Morale 42 No related terms Morale

From the sub-terms provided, those that related to citizenship education in schools were selected. In this case, they were moral education, moral instruction, moral values, moral reasoning and moral reflection. The main search term and its related sub-terms were included in the search. Where the list of related terms showed further related terms, these were also explored and often provided a search loop back to one of the original search terms.

A record of the search strategy is recorded in Appendix 2.2. This list gives the main search terms and related terms, including the scope notes provided in ERIC thesaurus. The ERIC search terms were used as the basis of all subsequent searches and, wherever possible, reference was made to the thesaurus of the specific database to match the database-specific terms to the ERIC search terms.

Searches were made of the contents lists and abstracts of those journals which are the key and leading publications for studies in the area of citizenship education. A list of these journals is recorded in Appendix 2.3.

2.2.3 Screening studies: applying inclusion and exclusion criteria

Searches of electronic databases produced a range of items that were downloaded into an Endnote database (DB1). Initial screening took the form of the application of search terms to electronic and online database. Items that appeared to meet the criteria were entered into database two (DB2). Parallel to this, journals and other sources of studies were handsearched (both printed and electronic versions of journals were available). Because abstracts, and in many cases the full text, were available at this handsearching stage, handsearched reports that appeared to meet the criteria were entered straight into DB2.

The inclusion and exclusion criteria were systematically applied to titles and abstracts of the studies where they were available.



The exclusion criteria were applied in a systematic way. This involved first excluding by date of publication (criterion c). If an item fell within the timescale covered by the review, it was then screened by study design and context. Only empirical studies were included (criterion e) and those that took place in a school context (criterion b). Items that met criteria b, c and e were then screened to see if they involved citizenship education (criterion a1) and learning or achievement (criterion a1 and a3).

All items that described schemes of work or curriculum programmes were excluded at this stage and empirically researched reviews were referenced as background reading. Finally, the items were screened by language, although all the studies in DB1 were written in English. Where the information required in order to apply the inclusion criteria was not given in a report or abstract, reviewers were over-inclusive and entered the report into DB2. Full text reports of all the papers in DB2 were obtained and re-screened for inclusion. All studies meeting the inclusion criteria were entered into a third database (DB3).

The inclusion criteria were also applied to the studies in the first review that focused on the impact of citizenship education on the provision of schooling. Studies meeting the inclusion criteria were included in the second review.

2.2.4 Characterising included studies

Studies that met the inclusion criteria were coded using the EPPI-Centre keywords (EPPI-Centre, 2003a), devised to be generic for educational research as a whole (e.g. relating to subject areas, population, setting). These are presented in Appendix 2.4.

The Review Group developed the following three sets of review-specific keywords: one set of keywords related to citizenship education and the others to learning and achievement (Table 2.2). Learning was understood as processes and achievement was understood as an outcome. Appendix 2.4 also gives definitions of these terms for the purpose of this review.



Table 2.2: Review-specific keywords

Citizenship Education Learning Achievement Moral and social responsibility Community involvement Political literacy Human rights education Education for diversity Spiritual moral social and cultural development Personal development Character education Emotional and social literacy Values education Service learning Active learning Conflict resolution Peer mediation Community participation Responsible action Civics Preparation for adult life

Creative thinking Critical thinking Meta-cognition Experiential learning Meaning-making Inter- and intra-personal awareness, including empathy Communication skills Collaboration Problem- solving/decision-making Values awareness

Cognitive outcomes (e.g. logical, linguistic, mathematical) Personal outcomes (e.g. inter and intra-personal development) Social outcomes (e.g. relationships with groups, societies, communities, organisations and the world) Moral and political outcomes (e.g. political literacy, political knowledge, ethical decision-making)

2.2.5 Identifying and describing studies: quality-assurance process

Two members of the Review Group applied the inclusion and exclusion criteria at each stage of the screening process and independently keyworded each of the studies and agreed the final version of keywording. The EPPI-Centre link team quality assured 10% of the application of inclusion / exclusion criteria to titles and abstracts and keyworded 10% of the included studies. At both stages, queries over interpretation of the criteria and codes were fed back to the group, along with screening and coding decisions.

2.3 In-depth review

2.3.1 Moving from broad characterisation (mapping) to in-depth review

Review Group members questioned the quality of the research in some of the items. Thus initially an additional screening process was added, which provided criteria to assess the quality of the reported research. Those studies not meeting these criteria were excluded.

All the studies included in the systematic map were screened for research quality. The following quality criteria were applied to all these studies.

1. The study meets reporting quality 1, namely that the research questions are stated: that is, the authors provide a succinct statement describing what the study is trying to explore/ describe/ discover/ illuminate and these research questions are stated in the abstract, in the introduction or background section or in a separate section entitled (for example, aims and objectives).



2. The study meets reporting quality 2, namely that at least some information is reported about the methods used in the study in each one of the following areas: the tools and people used to collect data; how the tools measure / capture the phenomenon under study; and the sampling and recruitment methods.

3. The study meets reporting quality 3, namely that at least some information is provided on the sample used in the study: that is, the units from which the data were collected, for at least two of the following characteristics: age, gender, socio-economic status, ethnicity, health status, or other relevant characteristics.

Following the screening for reporting quality, a second decision was made to focus on the impact of citizenship education on a particular outcome of learning and achievement; focusing on all aspects of learning and achievement was considered to be too complex to complete within the timeframe. Of particular interest to the Review Group was the relationship between citizenship education and one of the central tasks of education: that is, the development of knowledge, skills and understanding. Therefore the Review Group decided to focus on only those studies that reported on cognitive learning outcomes.

The in-depth review question was therefore as follows:

What is the impact of citizenship education on students’ cognitive learning outcomes?

To be included in the in-depth review studies had to:

• meet minimum quality reporting criteria

• focus on cognitive outcomes

2.3.2 Detailed description of studies in the in-depth review

Studies identified as meeting the inclusion criteria for the in-depth review, were analysed in-depth, using guidelines for coding and quality assessing educational research (EPPI-Centre 2003b) applying EPPI Reviewer, the EPPI-Centre’s review software. Appendix 3.1 gives an outline description by identification of study, country of study, focus of topic, curriculum area, type of study, type of citizenship education and aspects of learning and achievement. A detailed description of these studies is included in Appendix 4.2.

2.3.3 Assessing quality of studies and weight of evidence for the review question

Three components were identified to help in making explicit the process of apportioning different weights to the findings and conclusions of different studies. Such weights of evidence were based on the following:

• Soundness of studies (internal methodological coherence), based upon the study only (WoE A)



• Appropriateness of the research design and analysis used for answering the review question (WoE B)

• Relevance of the study topic focus (from the sample, measures, scenario, or other indicator of the focus of the study) to the review question (WoE C)

• An overall weight taking into account A, B and C (WoE D)

Across all four categories, studies could be judged high, medium or low. In order to receive a high judgement of overall weight of evidence (WoE D), a study had to receive a high judgement in each of the three previous weight of evidence categories of trustworthiness (WoE A), appropriateness (WoE B) and relevance (WoE C).

Although it can be argued that particular research designs may reduce the possibility of confounding factors more than others, the Review Group did not give greater weight to particular research designs because this field requires a range of types of research to illuminate a complex set of variables, and all were considered to be useful in producing evidence for practice.

The data were then synthesised to bring together those studies which answered the review question and met the quality criteria relating to appropriateness and methodology.

2.3.4 Synthesis of evidence

The nature of the review question and the structure for keywording classification provided the framework for the synthesis of evidence. It was clear from cross-referencing review-specific keywords that there were studies providing evidence across the range of types of citizenship education and range of aspects of learning and achievement. The in-depth review focused on those studies addressing and reporting on cognitive outcomes. The synthesis of findings was reported on, first by describing the evidence relating to cognitive outcomes, then by discussing the nature of the citizenship interventions involved, and finally by discussing the types of learning processes also reported on. This was in order to ensure a holistic report, which enables an interpretation of the evidence in an appropriately complex context of learning and teaching.

2.3.5 In-depth review: quality-assurance process

Data extraction and assessment of the weight of evidence brought by the study to address the review question was conducted by pairs of Review Group members who worked first independently, then moderated their decisions and came to a consensus. Members of the EPPI-Centre team worked with the Review Group in moderating data extraction and assessment of weight of evidence on 20% of the studies.


3. Identifying and describing studies: results

3. IDENTIFYING AND DESCRIBING STUDIES: RESULTS

This chapter provides details of the numbers of studies identified from the beginning of the searching process to the selection of the 35 studies, which constitute the systematic map, and the subset of 13 studies which were used for data extraction and synthesis. It also describes some of the general characteristics of those studies in relation to the review question, and describes the quality-assurance process.

3.1 Studies included from searching and screening Electronic databases searches identified 578 papers as relevant, which were entered into an EndNote database; 65 papers were identified from handsearching of journals and a further four papers were identified by personal contact from the previous review. This gave a total of 647 items in Database 1 (DB1). Of these 647 papers, 588 papers were excluded, with nine of these being kept for background reading. Thus 59 papers formed DB2 by meeting the inclusion criteria and were retrieved for full document screening. Of these 59 papers, 11 papers were unobtainable, leaving 48 papers available for screening by reviewers by careful reading of text, abstracts and full text and applying the keywords. Thirteen papers were excluded at this stage with two of these assigned as background reference material. This left 35 studies (reported in 35 papers) having met all the criteria for this review question to form DB3.

Figure 3.1 summarises the number of papers at each stage of the review.



Figure 3.1: Filtering of papers from searching to map to synthesis * Refers to the criteria cited in Appendix 2.1 ** Criteria are not mutually exclusive.

1. Identification of potential studies

Papers identified through database searching (N = 578), journal handsearching (N = 65)

and personal contact (N = 4)

Papers excluded N = 588

Abstracts and titles screened N = 647

Systematic map N = 35 studies (reported in 35

papers) (DB3)

Full document screened

N = 48

Papers to retrieve N = 59

Papers not

obtained N = 11

Criterion 2* N = 42

Criterion 3* N = 48

Criterion 4*N = 0

Criterion 5*N = 305

Papers excluded

N = 13

Criterion 1*N = 6**

Criterion 2* N = 1**

Criterion 3* N = 0**

Criterion 4*N = 10**

Criterion 5*N = 7**

In-depth review N=13 studies

Criterion 7* N = 16

Criterion 6* N = 6

Studies excluded

N = 22

Criterion 1*N = 193

2. Application of inclusion/exclusion criteria

* Key to criteria 1 not citizenship, not

learning, not achievement (a1, a2, a3)

2 not school (b) 3 pre 1988 (c) 4 not English language (d) 5 not empirical (e) 6 quality of reporting

insufficient 7 not cognitive outcomes

3. Characterisation

4. In-depth review



3.2 Characteristics of the included studies The following section summarises the characteristics of the 35 studies that met the inclusion criteria for the review’s systematic map (also see Appendix 3.1). It describes the range of study types, the types of citizenship education they address, the types of learning or achievement they measure, and other key characteristics.

In terms of study types, the majority of the 35 studies are evaluations either of naturally occurring interventions (N=16) or researcher-manipulated interventions (N=10); fewer studies were either descriptions (N=5) or explorations of relationships (N=4). The topic foci were predominantly teaching and learning (N=27) and curriculum (N=23) reflecting that the majority of studies were carried out within the school context. However within this studies were carried out across a range of curriculum areas and in both primary and secondary settings (N=13, N=21 respectively). Only one study focused solely on citizenship education in a single sex (all boys) setting (see Table 3.1). The majority of the studies were carried out in the USA (N=22), five studies were carried out in the UK and two from Australia. One study was identified from each of New Zealand, Portugal, Canada, Thailand, Ireland and Romania.

Considering the review-specific keywords, it is notable that citizenship education in these studies appears in a range of ‘forms’ or types, and the studies examine a range of types of learning and achievement. In terms of the former, moral and social responsibility (N=19) and community involvement (N=12), spiritual, moral, social and cultural development (N=12) and values education (N=12) are the most frequently represented types of citizenship education, while inter- and intra- personal awareness are the most frequently represented learning processes (N=25 and N=23 respectively). In terms of types of achievement the majority of studies measured social (N=29) and personal outcomes (N=27), while approximately half measured cognitive (N=17) and moral and political outcomes (N=15). These relationships are shown in more detail in tables 3.2. and 3.3.

Table 3.1: Details of selected studies (N=35 studies) Type of study (mutually exclusive) No. Description 5 Exploration of relationships 4 Evaluation: Naturally occurring 16 Evaluation: Researcher-manipulated 10 Type of citizenship education (not mutually exclusive) No Active learning 1 Moral and social responsibility 19 Community involvement 12 Political literacy 4 Spiritual moral social and cultural development 12 Character education 8 Education for diversity 4 Emotional and social literacy 10 Values education 12 Service learning 6



Conflict resolution 8 Peer mediation 4 Human rights education 4 Type of learning process (not mutually exclusive) No. Creative thinking 8 Critical thinking 12 Meta-cognition 2 Experiential learning 11 Meaning-making 16 Inter-personal awareness 25 Communication 14 Collaboration 12 Problem-solving/decision-making 11 Values awareness 15 Intra-personal awareness 23 Type of achievement outcome (not mutually exclusive) No. Cognitive outcomes 17 Social outcomes 29 Personal outcomes 27 Moral and political outcomes 15 Topic focus (not mutually exclusive) No. Classroom management 10 Curriculum 23 Organisation and management 7 Policy 1 Teaching and learning 27 Other topic focus 7 Curriculum focus (not mutually exclusive) No. Citizenship 14 Cross-curricular 2 Geography 1 Hidden 2 Literacy further languages 1 Art 1 ICT 1 Vocational Education 1 Literature 1 Music 1 PSE 2 Religious Ed. 2 Science 3 Other curriculum 9 The material does not focus on curriculum issues. 12 Institution (not mutually exclusive) No. Nursery School 1 Primary school 13 Secondary school 21 Other educational settings 7 Gender (mutually exclusive) No. Male only 1



Mixed sex 34 Source of study (mutually exclusive) No. Contact 1 Handsearch 13 Unknown 2 Electronic database 19



Table 3.2: Type of citizenship education by learning process (N=35 neither category mutually exclusive)

X-axis: What type of citizenship education does the study examine? Y-axis: If the study measures the impact on learning, what kind of learning is being studied?

Mor

al a

nd s

ocia

l re

spon

sibi

lity

Com

mun

ity

invo

lvem

ent

Polit

ical

lite

racy

Spiri

tual

mor

al

soci

al a

nd

cultu

ral

deve

lopm

ent

Educ

atio

n fo

r di

vers

ity

Cha

ract

er

eca

tion

du Emot

iona

l and

s

cy

ocia

l lite

ra

Valu

es

educ

atio

n

Serv

ice

lear

ning

Con

flict

re

solu

tion

Peer

med

iatio

n

Hum

an ri

ghts

ed

ucat

ion

Act

ive

lear

ning

Creative thinking 6 3 0 4 1 4 2 5 2 2 2 1 0 Critical thinking 8 2 1 6 0 5 3 6 1 1 1 1 0 Meta-cognition 2 1 0 2 1 1 2 1 1 0 0 0 1Experiential learning 5 7 2 4 1 1 3 2 4 1 0 0 1

Meaning-making 10 7 0 8 2 5 4 7 6 1 1 3 1Inter-personal awareness 14 9 3 10 4 5 9 9 5 8 4 3 0

Communication skills 7 2 1 4 1 3 6 5 3 6 3 2 0

Collaboration 5 3 2 4 1 2 5 5 3 5 2 2 0Problem-solving/decision-making

5 2 0 1 1 3 4 3 2 6 4 20

Values awareness 8 2 2 7 2 3 5 9 3 4 3 4 0

Intra-personal awareness 12 7 3 7 3 6 6 8 3 7 4 2 0



Table 3.3: Type of citizenship education by achievement (N=35 neither category mutually exclusive)

X-axis: What type of citizenship education does the study examine? Y-axis: If the study measures the impact on achievement, what kind of achievement is being studied?

Mor

al a

nd

soci

al

resp

onsi

bilit

y

Com

mun

ity

invo

lvem

ent

Polit

ical

lit

erac

y

Spiri

tual

m

oral

soc

ial

and

cultu

ral

deve

lopm

ent

Educ

atio

n fo

r di

vers

ity

Cha

ract

er

eca

tion

du Emot

iona

l and

s

cy

ocia

l lite

ra

Valu

es

educ

atio

n

Serv

ice

lear

ning

Con

flict

re

solu

tion

Peer

m

edia

tion

Hum

an ri

ghts

ed

ucat

ion

Act

ive

Lear

ning

Cognitive outcomes 11 5 1 9 1 5 3 7 4 2 2 3 0

Personal outcomes 15 9 3 8 4 6 9 8 6 7 3 3 1

Social outcomes 16 11 3 9 4 5 10 10 5 8 4 3 1

Moral and political outcomes

10 5 3 8 3 4 6 8 4 3 2 3 1



3.3 Identifying and describing studies: quality- assurance results The application of inclusion and exclusion criteria to reports in DB2 were applied by two reviewers and moderated by the EPPI-Centre team, as was a sample of studies from the electronic searching. The search process and terms were reviewed thoroughly by the whole team and by members of the EPPI-Centre. Standard EPPI-Centre keywords and review-specific keywords were applied to all studies in DB3 by two members of the Review Group and 10% of these were moderated by the EPPI-Centre team. The moderation process revealed few disagreements in screening and coding which were easily resolved through discussion.


4. In-depth review: results

4. IN-DEPTH REVIEW: RESULTS

This chapter describes the evidence drawn from the 13 selected studies. This evidence is synthesised in two sections. First, it describes the evidence of the impact of citizenship education for the cognitive outcomes of the studies, the characteristics of the citizenship interventions, and then the learning processes and other achievement measures which were reported on in these studies. Second, it identifies key themes emerging from across the range of evidence from the studies.

4.1 Selecting studies for the in-depth review

Having mapped the research literature, a decision was taken only to include studies which report on cognitive outcomes and those with sufficient quality of reporting, thus reducing the number of studies in the in-depth review to 13.

4.2 Comparing studies selected for in-depth review with the total studies in systematic map Like the studies in the systematic map, the 13 studies in the in-depth review were characterised by a breadth of types of citizenship education and by a range of learning processes and learning outcomes. This is represented in Appendix 4.1, which compares the studies in the systematic map and the in-depth review.

4.3 Further details of studies included in the in-depth review The full details of the final studies selected for in-depth review are summarised by aims, research questions, study design, type of study, findings and conclusions and weight of evidence in Appendix 4.2.

Of the 13 studies in the in-depth review, five were judged to be of high trustworthiness (WoE A) on their own terms, six were judged to be of medium trustworthiness and two were judged to be of low trustworthiness; four were judged to be of high appropriateness for answering the review question (WoE B), eight were judged to be of medium appropriateness and one was judged to be of low appropriateness; five were judged to be of high relevance for answering the review question (WoE C), seven were judged to be of medium relevance and one was judged to be of low relevance. Overall (WoE D), there was one study that was rated as contributing a high weight of evidence to answer the specific question of this systematic review, 11 studies which were rated as contributing a medium weight of evidence and one study which was rated as low weight. In order to receive a high judgement of overall weight of evidence (WoE D), a study had to receive a high judgement in each of the three previous weight of evidence categories of trustworthiness (WoE A), appropriateness (WoE B) and relevance (WoE C). Table 4.1 demonstrates this.



Table 4.1: Classification of studies by review-specific weight of evidence and trustworthiness

Study A. Trustworthiness

B. Appropriateness

C. Relevance

D. Overall weight

Beyer and Presseisen (1995)

Medium Medium Medium Medium

Black and Goldowsky (1999)

High Medium Medium Medium

Clare et al. (1996) Low Medium High Medium Day (2002) High Medium Medium Medium Deakin Crick (2002) High High Medium Medium Faubert et al. (1996) Medium High High Medium Garcia-Obregon et al. (2000)

Medium High Medium Medium

Laconte et al. (1993)

Low Low Low Low

Melchior (1999) Medium Medium High Medium Polite and Adams (1997)

Medium Medium Medium Medium

Russell (2002) High Medium Medium Medium Tibbitts (2001) Medium Medium High Medium Wade (1994) High High High High

4.4.Synthesis of findings

4.4.1. Introduction

All 13 studies included in the in-depth review have been data extracted and keyworded and identified as having cognitive learning outcomes. In order to answer the review question, the findings of 13 studies will be reported initially in terms of evidence of cognitive outcomes, such as higher order, critical and abstract thinking, conceptual and creative thinking with the application of concepts and principles to situations, meta-cognition, meaning-making and making connections between concepts and experiences, communication skills, problem-solving/decision-making, the award of grades and academic achievement. Other significant outcomes, such as social and personal outcomes, are reported on secondarily where relevant, because, while not being cognitive, these outcomes are recognised as making a fundamental contribution to the cognitive learning and achievement of the whole person.

These papers are now discussed in turn, in terms of weighting, cognitive outcomes findings and learning processes.



4.4.2. Summary of findings

High weight of evidence

In a small-scale study rated high for weight of evidence in relation to the review question by Wade (1994), the aim was to determine the cognitive, contextual and motivational factors influencing conceptual change processes as students attempted to make sense of the abstract concept of human rights. In this retrospective analysis of an earlier ethnographic project, the study discusses limitations and successes in students’ learning within the context of change and motivation research. The research was conducted in a rural elementary state school in New Hampshire, USA, in a classroom of 17 students aged 9 to 11 years old. It reports on a series of activities and teaching strategies employed by the researcher and class teacher, which were designed to expand students’ understanding of the abstract notion of human rights. Activities such as art, drama and writing were used, as was discussion, co-operative learning, role-play and simulations, with a special emphasis on human rights through writing a personal story. Weekly class conferences were held at which students could express themselves and enter into dialogue to address their rights and responsibilities in the classroom. Through these processes, the researchers attempted to engage students in the learning processes of conceptual thinking and meaning-making where the students could build bridges and make connections between their own experiences in the classroom and relevant human rights.

A number of findings from this study supported prior research, indicating that upper elementary students are concerned about rights and fairness in their daily lives, yet have little understanding of the concept of human rights initially. The study found that the understanding of the process of conceptual change is important as this can influence students’ understandings of human rights and cognitive engagement. It reports that those students who had accurate schemata and task mastery goal orientation to learn, were the most successful as this supported their further knowledge development about human rights. Another finding was that some students were motivated by their own life experiences to learn about relevant human rights and that other students, in the face of conflicting information, relied on strongly held patterns of thinking based on their prior knowledge and experiences. Therefore, the report suggests that personal agendas, interests and cognitive engagement influenced students’ understanding of human rights and points to the role of emotional salience in fostering motivation to learn and cognitive engagement as an important area for further research. The study suggests that the structure of learning activities affected motivation and cognitive engagement among students, who were in general highly motivated during the unit activities.

The study concludes that, in order for teachers to promote cognitive engagement, they must challenge students’ misconceptions and teach them to illuminate conceptual conflict by structuring discussions so that all students will justify and explain their views and reconsider their ideas in light of new information. The study recommends that, for this to occur, it will be necessary to change traditional teaching methods and teachers themselves.

Medium weight of evidence

In a nationwide study of 17 schools and over 1,000 participants in the USA, a medium-rated study by a team of researchers (Melchior, 1999) evaluated the impact on student learning, knowledge and skill development, and short-term and



one-year effects of the service-learning programme, entitled ‘The Learn and Serve America Program’. The study evaluated only those programmes which were considered to have the characteristics of a well-designed service-learning programme in order to disseminate features of good practice. Students in the programme were involved in a relatively intensive programme of hands on and face-to-face involvement with substantial hours of direct service, working as tutors or teacher’s aides in nursing homes or homeless shelters. This work was linked to the academic curriculum through literature reading in English, with students participating in formal and informal group discussions about their experiences, keeping journals, and producing other forms of weekly written reflection exercises, such as essays, research papers, making group presentations, and discussion of such analyses.

In terms of cognitive outcomes, the educational impact of the programme was measured through individual course grades. It was found that there was a significant positive impact on students’ mathematics grades, with incremental, but not significant, gains in English grades. There was some evidence among participants from ethnic minority backgrounds of stronger impacts on measures of academic achievement.

Personal and social outcomes were reported on with findings of a positive short-term effect on participant civic attitudes and involvement in volunteer service; however, there was little evidence of longer-term impacts one year later. The study reports that effects were significantly greater among non-white and the educationally disadvantaged. The programme was considered value for money as, on average, participants provided services valued at nearly four times the programme costs. Statistical evidence also demonstrated positive experience of participation, impact on student career choice, better understanding of community and the benefit that students learned more than if in a class.

The report concludes that, while there are significant benefits for schools, communities and students involved in service learning programmes, it is not a vehicle for large-scale change. However, the study reports that good quality and well-designed programmes do make a difference and are more likely to have significant impact. The study found strong impacts, which made students more aware of community and engagement in school leading to improved school performance, reducing the likelihood of middle-school students engaging in risk behaviours with a strong case for the programme as a tool for civic and educational development of students.

The medium-rated paper by Beyer and Presseisen (1995), reports a small-scale experimental intervention study. The aims of this study were to evaluate the impact of a six week intervention history project in a USA urban setting involving 36 students, with 20 students participating in the programme and 16 non-participants in the control group. The project was an interdisciplinary moral education programme and was undertaken to address two cognitive outcomes of the programme’s overall goals, which were:

• to increase middle- and high-school student knowledge of the period of Nazi totalitarianism, the Holocaust, causes and roles played by various people and groups

• to increase student ability to make connections between historical events and current issues of prejudice, racism and hatred and their own experiences and choices



The findings of the study demonstrate that the experimental group performed significantly better for increases in knowledge of historical content, reasoning about human relationships, and the implications of their own actions. There was a large and significant gain in factual knowledge of totalitarianism but there were variable responses from the students with evidence that some students did not demonstrate high levels of understanding. Other evidence indicated that student opinions remained tied to personal experiences rather than historical knowledge, with students making meaningful connections between the situation of the Jews and the slave trade. The report concludes with the authors’ awareness of the limitations of the study with the recognition that: some students were unable to use written language to express ideas and thoughts; the topic created a high level of moral engagement and involvement for both the experimental and control groups; and it was a relatively short study.

The medium-rated study by Black and Goldowsky (1999) was a large-scale study involving data collection from 745 participants in Massachusetts, USA. This study was an investigation of a naturally occurring intervention that used theatre as pedagogy in the teaching of science and its impact on student learning, ability to reason and understanding of social and moral implications of sensitive scientific issues. The study attempted to consider theatre as an alternative way to teach controversial issues, such as the Human Genome Project, with students watching a play consisting of a dialogue between a male and female actor in order to provide a balanced viewpoint of the topic. Audience participation was encouraged to explore the topic through discussions and feedback was collected by way of a questionnaire with open-ended questions.

The study reports on cognitive outcomes, such as meaning-making, conceptual and abstract thinking, reasoning and communication skills. Students thought the play was educational and had a role in educating, informing and teaching science. Another finding was that, after seeing the play, students were better able to reason about the ways the issue of the Human Genome Project might influence and impact their lives. Students found the play to be a thought-provoking, relevant and engaging way of presenting such an issue and allowed them the opportunity to make connections with the actors, particularly those of their own gender. They also found that the play initiated their ability to link the complex scientific and moral issues raised by the play to personal and societal matters. The authors conclude that (a) theatre should be used to make learning about science more meaningful as it helps to organise students’ thoughts and help with successful reasoning about social, moral and ethical issues, and (b) more attention should be given to the use of different strategies to encourage greater student participation.

A study carried out by Clare et al. (1996) examined how teachers could implement extended discourse in the literature classroom in order to assist students to develop the cognitive outcomes of higher order comprehension, and to facilitate and enhance students’ abilities to develop, express and debate complex moral and social themes in literature texts. In this medium-rated evaluation of a naturally occurring intervention, instructional conversations of one teacher in a class of fourth-graders in the USA were studied. Previous moral development research demonstrated that more extended discourse is a vital element in effective moral education; however, the difficulty of implementing such discourse in the classroom is seldom discussed. Learning processes included conversational discussions whereby the teacher first used direct teaching methods where necessary, providing the students with the pertinent background and relevant schemata for understanding a text, which would then be followed by a discussion. Extended student contributions were elicited by skilful



conversational questioning and would promote students’ use of texts, pictures and reasoning to support their argument or position, where there may be more than one correct answer. The evidence suggested that this occurred best in a challenging but non-threatening atmosphere, where the teacher has created a zone of proximal development balanced by a positive affective climate where students can construct and negotiate the meaning of the text; where the teacher does not hold the exclusive right to determine who talks; and where students are encouraged to volunteer or decide to speak in turns.

In the authors’ analysis of the study’s findings, it is reported that young students initiate discussion of values implications of texts they have read if they have increased opportunities for connected discourse. Another finding was that students in groups that engaged with moral issues were four times more likely to mention more subtle or problematic effects of friendship in their essays; there was also more evidence of an impact in students’ essays where the teacher used conversational discussion than where those students were taught conventionally. Traditional teaching methods, such as simply reading the story, was found to be insufficient as a key requirement is instruction by assisted performance with skilful questioning by the teacher to help bring students their focus. By this method, students were better able to comprehend problems between characters and better able to get more than basic points, moving beyond superficial understandings to explore moral and emotional quality.

The study concludes that the highly interactive and challenging discussion approach of instructional conversation, together with a focus on moral dilemmas, reveal that moral education can be a useful tool for educators attempting to enhance student comprehension. However, the study acknowledges teachers need to develop the technical ability to generate and lead such discussions and present students with topics that are appropriate and compatible with students’ moral development and reasoning. Teachers must be versed in the use of open-ended questions, clarifying, summarising, building on student contributions and encouraging students to respond to each other.

In a medium-rated study by Day (2002), based in England, an evaluation of a naturally occurring, small-scale intervention explored the impact of a theatre workshop on students’ learning about the dilemmas of refugees and the homeless. The workshop’s reality was intensified because the actors had experienced homelessness and on account of the presence of refugee children at school. The study demonstrated the cognitive outcomes of students’ reasoning and communication skills of discussion, dialogue and debate through the provision of the co-operative learning environment of the theatre workshop. The workshop was seen to provide a powerful and interactive learning experience with group discussions and opportunities for student intervention. Another cognitive learning outcome of the study was that students were able to transfer and make meaning of their personal responses to the plight of refugee children in school to human experiences in general. The workshop also provoked a desire among the students to take action in real life about the issues explored.

Deakin Crick’s (2002) medium-rated study explored relationships between a values education intervention in an English school and the impact on pupils’ learning, and general and specific subject knowledge. The researchers worked with teachers to identify opportunities to implement values education across the curriculum that required cognitive learning outcomes from students, such as higher order thinking, and making meaning and connections between the school curriculum and their real lives. These strategies also anticipated affective



outcomes, such as encouraging a positive disposition towards lifelong learning, valuing equality in the needs of others and deepening interpersonal relationships between teachers and students.

Data were collected before and after the intervention via Kelly’s Repertory Grid (1955) where participants developed their own constructs in responses to questions. For students and teachers alike, the most significant family of constructs related to cognitive outcomes of learning, achievement and excellence. The study found that students’ thinking skills went beyond what normally occurs in lessons and that the intervention enabled students to make sense and meaning out of their learning. The researcher reported that dialogue was the most appropriate medium for students to use to explore making connections between the school curriculum and their experiences, and the findings emphasised that the importance of quality discourse cannot be underestimated. Another finding was that the quality of thinking and talking was significant to the whole process of enhanced learning.

In different subjects, interventions, discussions, role-play and questions, which were all related to lesson content, led to correlations between the way students dealt with values prior to the intervention and after. This led to the study concluding that values interventions were evident in giving rise to cognitive and affective learning outcomes. The use of values education, based on questioning and dialogue, encouraged students in the processes of reflective searching for deeper meanings to events and issues, and the consideration of the abstract concepts of justice, stewardship, respecting others, sympathy, empathy, sharing of ideas, listening skills, trust and forgiveness. Values education impacted on the way students worked with each other in groups and as a tool to transfer learning.

The study concludes that values education does not just concentrate on cognitive outcomes or should be limited to one-off lessons, as opportunities for values education arose in most subject areas across the curriculum. The study confirms and supports the importance of a positive holistic relationship between teachers and students. As the study addressed the whole person as a learner, this resulted in teachers relating differently to students; in turn, this led to teachers considering their teaching to be more purposeful and provided an additional dimension to their subject.

Faubert et al.’s (1996) medium-rated USA study aimed to explore whether an intervention action learning programme, using the ‘Deliberate Psychological Education’ (DPE) theory model, called ‘Getting on the Right Track’, would promote the cognitive outcome of concrete to abstract thinking and the affective outcome of ego/self-concept development among African-American students. This study was a quasi-experimental design with ninth- and tenth-graders (ages 14 to 16 years), with an experimental group of 80 students receiving the intervention and a comparison group of 76 students who did not. The intention of the intervention programme, which ran for five months, was to provide emotional and cognitive support with the purpose of enhancing growth over a significant amount of time.

The intervention focused on learning to think like a scientist with a group of ninth- and tenth-grade students learning science by doing a science research project and a role-taking activity. The topics were selected on the basis of relevance to African heritage and the contribution of African-American scientists. In learning the scientific method (i.e. hypothesis testing and inductive and deductive reasoning), the students were exposed to a wide range of ideas and direct experience in applying the methods of science. The content was historically



relevant and the process was a combination of action and reflection. For the role-taking activity, students were placed in the communal role of helping a peer with the tenth-grade students trained as peer helpers to assist their ninth-grade colleagues, by active listening and empathic responding, to accomplish the following cognitive learning outcomes: to develop an understanding of what it means to say, ‘Science is a way of making meaning of our history and experience’; to choose a research topic; to search the literature and synthesise the information by writing a research question; and to report findings orally and in writing.

The findings reported the experimental groups receiving DPE were found to demonstrate significantly greater gains in cognitive abilities (controlling for initial cognitive skills), with the main effect for males in the experimental groups being the greatest. In addition to this, the experimental groups receiving DPE were found to demonstrate significantly greater gains in ego/self-concept development (controlling for initial ego stage), with the main effect for females in experimental groups being the greatest. Therefore, the findings indicate statistically significant positive changes in both formal operations of concrete to abstract thinking and self-concept development in the experimental groups.

Further qualitative findings support and cross-validate the empirical outcomes of the quantitative findings for the affective domains of increasing participation, motivation levels and heightened involvement in the schools, with students taking their responsibilities seriously, working diligently, and presenting their research projects at the science festival with poise and competence. The positive attitude and behaviour of the students provided evidence for these African-American rural adolescents’ growth in the cognitive and ego domains.

In conclusion, the study found two major empirical measures supported positive outcomes from the ‘Getting on the Right Track Project’. On the assessment of cognitive growth from concrete, literal thinking toward the ability to think abstractly and scientifically results indicated significant gains. This shift from concrete to abstract thinking usually takes a long time (four to five years; Renner et al., 1976); thus these findings support the effectiveness of the DPE. The results suggest that it is critical for African-American rural youth to have an opportunity to develop higher order intellectual skills and the study suggests that the current programme offers evidence that this DPE intervention can be considered a step in the desired direction for the development of cognitive growth. On the second measure, the instrument of self-concept/ego development, the results have similar indications. At higher levels of ego development, adolescents begin to demonstrate greater autonomy, more reliance on self, a firmer sense of identity and greater reliance on independent problem-solving (Loevinger, 1976).

The authors debate about which is more important – building academic skills or promoting self-reliance – and view the solution as interactive. By helping students develop their thinking skills and simultaneously helping connect these to their valued heritage, they do not have to choose which is more important; both are equally relevant. Self-reliance and a strong ego by themselves cannot make up for cognitive deficit, and cognitive growth cannot generalise to self-reliance and a strong ego. In fact, the authors would urge that future innovative educational programmes seek such dual goals as major objectives as it is clear, based on qualitative and subjective observation, that the students in the experimental classes became more potent learners and more self-confident as students.

Another naturally occurring intervention study judged to provide medium weight found a significant positive impact on cognitive and affective outcomes following a



school-based service-learning programme in a Texas school. The study by Garcia-Obregon et al. (2000) evaluated the ‘Community Connection Program’ to determine the impact on four areas of learning of students and the area of perception for teachers. The researchers gave questionnaires to 120 students and 19 teachers, specifically to find out if students participating in such a scheme show improved cognitive outcomes through achievement gains in reading and mathematics, and in the affective domain areas of higher school attendance rates, lower incidences of misbehaviour and a positive perception of self, school and community. As the scheme suggested that involvement could contribute to student learning and growth, the scheme was integrated into all curricular areas. Teachers facilitated student development of knowledge and skills and students:

• prepared for service

• undertook service

• reflected on and recognised their experiences through group work

The study findings were that programme participants made increased gains in academic achievement, with a significant gain in Texas Learning Index scores and also a decrease in the failure rate on the Texas Assessment of Academic Skills test for reading and mathematics. There was an increase of 17.1% in the pass rate to 92.9% for the intervention cohort. Other findings for the cohort were:

• decreases in number of absences

• lower number of referrals for misbehaviour

• students reported feeling better about themselves as the programme helped them get along with and care about others

• teachers also had strong support for and positive perception of the scheme

The authors concluded there was sufficient evidence to support the view that the ‘Community Connection Program’ was effective in the areas identified for students and teachers. The quantitative and qualitative data supported the conclusion that the programme impacted on students’ academic achievement, attendance, behaviour and self-perception, and that teachers viewed the programme positively.

A study that was medium-rated in terms of the impact on student learning and achievement was that by Polite and Adams (1997). In order to determine whether a series of Socratic seminars was effective for the development of student’s cognitive outcomes, the researchers randomly selected a sample of 34 students and nine teachers in one USA middle school. (Socratic seminars are a form of scholarly discourse conducted in grade K–12 classes that seek to create dialogue rather than debate.) The cognitive outcomes were as follows:

• higher order, abstract and critical thinking skills

• engagement in fluent communication, responsible speech and contribution to class discussion

• writing, analysing, classifying, comparing, hypothesising, inferring and generalising skills



• exploratory experiences for all students

• increased student ability and achievement

The findings support the usefulness of the Socratic dialogue process in promoting higher order thinking, such as meaning-making and linking the topics to their own life experiences, and the development of appropriate conflict resolution strategies and skills.

With regard to higher order thinking skills, approximately 80% of the student sample engaged in at least intermittent meta-cognitive activity or Piagetian formal operational activity. This was unusual, as sixth-graders would usually be functioning at the level of Piagetian concrete operations. This finding applied to both boys and girls, although the girls scored slightly higher than the boys. Approximately 40% of the students attempted to use meta-cognitive analytical skills in their life experiences, with 60% perceiving meta-cognitive skills as valuable outside the seminar environment.

The level of conflict resolution expertise demonstrated by the students in the study was significantly higher than would normally be expected from a middle school population with 66% of girls and 50% employing advanced conflict resolution strategies.

Other findings indicate affective outcomes, with an increase in interest in learning as 50% of students strongly preferred the seminar environment to the regular classroom: students felt they accomplished more; they felt their thoughts and feelings were taken seriously in the seminars; and, to them, seminar topics were always more interesting. Students made comments about a difference in their peer behaviour with fellow students seeming more polite in seminars and more of the class participated actively. A minority of students disliked the setting and expressed a preference for structure and authority figures, saying they could only learn when teacher was dispensing information.

The researchers conclude that there is evidence that Socratic seminars have the potential to increase the cognitive and social functioning of middle school students. Student interviews indicate that Socratic seminars provide students with ample opportunities to develop a number of cognitive and social skills, commonly regarded as key goals for children at the middle-school level. There was significant evidence that seminar discussions were effective in engaging students in tasks that called on their meta-cognitive and critical thinking abilities, while simultaneously developing both conflict-resolution skills and an increased respect for the opinions and feelings of their peers.

In a medium-rated study by Russell (2002), the cognitive processes of higher order thinking; understanding; making judgements and decisions; deduction, clarification, making connections, distinctions and generalisations; and reflective thinking were all helped through student interactions, discussions and dialogical processes. This study examined children’s moral consciousness and the part played by discussion in helping to foster their moral development. The setting for this naturally occurring evaluation was Ireland, with a total of 59 participants in two groups. Twenty-nine children aged 7 and 8 were in one group and 30 children aged 11 and 12 were in the other. The children were participating in a community of inquiry, known as ‘Thinking Time – Philosophy with Children’, with lessons based on discussion, less influence from the teacher and reflective thinking by the children.



Findings indicate that the interactive dialogue process helped children to refine their ideas from broader definitions, with older children in particular using more deductive reasoning, making more distinctions and demonstrating a superior ability to articulate, which enabled higher levels of reflection, with different levels of thinking evident between the older and younger students.

Other findings are that, through listening to others and respecting others, students exhibited a deepening ability to reciprocate and respond to others, which is central to moral judgement and action; students were aware they should do what is right and take responsibility for their decisions which demanded a sense of courage with both boys and girls using impartial and empathic reasoning. The research indicated that giving children a ‘voice’ boosted their confidence because they felt what they said mattered and was listened to, with higher levels of participation from children who experience learning difficulties in other areas and whose thinking was challenged and ‘assisted’ by the more able children.

The report’s overall conclusion is that dialogue has a role to play in fostering reflective thinking, understanding, tolerance and respect for others, and demonstrates the importance of student voice.

The medium-rated study by Tibbitts (2001) demonstrated a link between instructional methodology and the emergence of student ability to hold multiple ideas and grasp abstract concepts related to the state and governance. This naturally occurring evaluation of the impact of a Civic Education project was undertaken in three schools in different locations in Romania at a time of democratic transition. The evaluation was a quasi-experiment with data collected from 113 children in the ‘treatment group’ who received experimental Civic Education texts, and 109 children in the ‘control group’ who received Ministry of Education civic texts.

The ultimate goal of the Civic Education project was to encourage pluralistic and critical thinking, self-expression and dialogue in the classroom as a tool for an organic form of long-term democratic transition. These experimental texts were regarded as unique in Romania when they were introduced at this time as they emphasised fundamental democratic values, rights of the child and practices, such as dialogue, critical reflection, individual work and group work. The study found that students were encouraged by ‘more learner-centred’ instructional techniques, such as discussion groups to learn and analyse concepts in their social and political worlds. Evidence indicated significant gains in the experimental group’s understanding of the notions of citizenship, voting and participation in public life which was greater than in the comparison group, which had received the conventional Ministry texts.

Low weight of evidence

In the report by Laconte et al. (1993), which was low rated in terms of contribution of weight of evidence to this review, the researchers undertook a small-scale experimental study to investigate the impact of a developmentally appropriate affective education programme on grade achievement and self- concept of emotionally disturbed students who were identified to be at risk of dropping out of school before completion of high school. An intervention programme, called ‘Thinking, Feeling, Behaving: An Emotional Education Curriculum for Adolescents’ (Vernon, 1989), was followed with 13 students randomly allocated to the experimental group, which met for 15 weeks and included topics such as problem-solving, communication and relationships. Ten students were allocated to the control group and did not receive the programme.



This study found no significant difference between the experimental and the control groups in grade scores and no difference for self-concept scores. These findings are consistent with other carefully designed studies of programmes where the focus is on self-concept, which found no significant positive findings (Strein, 1988). In conclusion, the researchers discuss possible factors for these results and make six observations:

• lack of time in teaching sessions

• ‘outside circumstances’

• characteristics of the student group, such as low self-esteem (not measured by pre-test)

• small sample size

• lack of effective implementation of the programme

• lack of effect of this programme as compared with other possible programmes

Key themes emerging from the findings

An in-depth analysis was conducted by two reviewers to identify emergent key themes from the findings of the 13 studies. This process resulted in the identification of five categories of cognitive learning outcomes and five learning processes. The evidence suggests that cognitive learning outcomes are not arrived at in isolation from personal, social and moral/political outcomes, and that these outcomes are informed by the nature and quality of the learning in which the student participates.

Although in this review a decision was taken to focus on 13 studies that report on cognitive learning outcomes, this subset of 13 studies also report evidence relating to other types of outcomes and a range of learning processes that cannot be reduced to a simple cognitive domain. The evidence suggests that it is the interrelatedness of affective, cognitive, and volitional learning processes, interacting together in a cohesive and well planned citizenship education programme (Melchior, 1999; Deakin Crick (2002), which has an impact on the achievement of the learner as a whole person, who then displays greater self-confidence and is a potent learner (Faubert et al. (1996).

An overarching finding is that the effects of these variables are difficult to study independently of each other because they function together more in the manner of a complex ‘ecology’ rather than as a set of discrete entities. Thus the evidence suggests that learning and achievement in citizenship education is a complex domain, which cannot be reduced to a single variable. Student learning and achievement is associated with the nature and quality of the pedagogy of the subject (learning, teaching and the nature of the curriculum). The following discussion will deal with these factors together to reflect this complexity.

Emergent cognitive learning outcomes

The five categories of cognitive learning outcomes drawn from across the 13 studies (subsequently referred to as a group of developing holistic approaches to achievement – see Chapter 5) are as follows:



• meaning-making


• higher order thinking skills

• academic achievement


Table 4.3: Emergent cognitive learning outcomes: findings from in-depth review Cognitive learning outcomes

Characteristics Study

Meaning-making Making connections, making sense of content, apply knowledge to personal experiences – in own world and society

Beyer and Presseisen (1995), Black and Goldowsky (1999), Clare et al. (1996), Day (2002), Deakin Crick (2002), Melchior (1999), Polite and Adams (1997), Russell (2002), Tibbitts (2001), Wade (1994)

Understanding and reasoning

Improved understanding, reasoning, decision-making, problem-solving

Beyer and Presseisen (1995), Black and Goldowsky (1999), Clare et al. (1996), Day (2002), Deakin Crick (2002), Faubert et al. (1996), Melchior (1999), Polite and Adams (1997), Russell (2002), Wade (1994)

Higher order thinking skills

Meta-cognition, abstract, complex, critical, analytical, reflective and conceptual thinking

Black and Goldowsky (1999), Deakin Crick (2002), Faubert et al. (1996), Polite and Adams (1997), Russell (2002), Tibbitts (2001), Wade (1994)

Academic achievement

Improved grades, increased achievement/ability, cognitive engagement/ functioning

Beyer and Presseisen (1995), Garcia-Obregon et al. (2000), Melchior (1999), Polite and Adams (1997), Wade (1994)


Self-expression, debate, listening

Black and Goldowsky (1999), Day (2002), Russell (2002), Wade (1994)

Emergent learning processes

The following learning processes were identified from the findings as making a significant contribution to the five categories of cognitive learning outcomes.

• engagement







Table 4.4: Emergent learning processes: findings of in-depth review Learning processes


Engagement Increased opportunities to participate and interact, explore, student ‘voice’, self-expression, felt listened to and what they said mattered; student self-responsibility for own decisions, choices and own work; student self-reliance, student engagement

Black and Goldowsky (1999), Clare et al. (1996), Day (2002), Deakin Crick (2002), Faubert et al (1996), Garcia-Obregon et al. (2000), Melchior (1999), Polite and Adams (1997), Russell (2002), Wade (1994)

Promoting discussion

Discussions, teacher-learner dialogue, conversational discourse, open-ended questions, communication and listening skills, co-operative learning environment, holistic teacher-learner relationship, learner-learner relationship

Black and Goldowsky (1999), Clare et al. (1996), Day (2002), Deakin Crick (2002), Melchior (1999), Polite and Adams (1997), Russell (2002), Wade (1994)

Learner- centred teaching

Teacher technical skills to lead discussions, clarify and summarise, knowledge of student ability level and appropriateness of teaching material, learner-centred approaches/facilitators versus didactic traditional methods insufficient/teachers no longer reciters of wisdom; students learn through doing/relating content to own experiences, essays, research papers, writing and making presentations, group and individual work, journal-keeping, weekly reflection exercises

Clare et al. (1996), Day (2002), Deakin Crick (2002), Faubert et al (1996), Melchior (1999), Russell (2002), Tibbitts (2001), Wade (1994)

Meaningful curricula

Real-life contexts, connecting historical/abstract with real-life experience of students, theatre, role-play, festival presentations, simulation, drama, art and writing activities, seminar forums, formal/informal settings, programme length, quality of programme/delivery makes a difference

Beyer and Presseisen (1995), Black and Goldowsky (1999), Day (2002), Deakin Crick (2002), Faubert et al. (1996), Laconte et al. (1993), Melchior (1999), Polite and Adams (1997), Russell (2002), Tibbitts (2001), Wade (1994)



Learning processes


Developing personally

Improved attitudes, behaviour, better attendance, positive experiences, positive self-image and self-confidence, care and respect for others, ability to respond, conflict-resolution skills, increased interest in learning, beneficial impacts non-white and educationally disadvantaged students, and students with learning difficulties

Clare et al. (1996), Faubert et al. (1996), Garcia-Obregon et al. (2000), Melchior (1999), Polite and Adams (1997), Russell (2002)

4.4.4 Discussion of findings: cognitive learning outcomes

Meaning-making

There is strong support from the study findings that citizenship education can engage students to seek (cognitive) understanding for the meaning of their personal stories and experiences when learning about lesson content and knowledge of others’ situations (Beyer and Presseisen, 1995; Black and Goldowsky, 1999; Day, 2002; Polite and Adams, 1997). There is evidence that students’ motivation to participate increased when topics were pertinent and relevant to their own experiences and they could make connections between the content and experiences in their personal lives (Wade, 1994; Polite and Adams, 1997). Students were able to apply this knowledge of others’ situations to make sense of:

• their own personal and social worlds making learning more meaningful (Black and Goldowsky, 1999; Deakin Crick, 2002)

• the implications of their own actions (Beyer and Presseisen, 1995)

• how they regarded people who may be in similar challenging situations (Day, 2002)

Meaning-making was linked to innovative learner-centred media, such as the use of theatre, role-play, simulations, conversational instructional discourse, discussion, journal-keeping and reflection exercises (Black and Goldowsky, 1999; Clare et al., 1996; Deakin Crick, 2002; Faubert et al., 1996; Melchior, 1999; Wade, 1994). The findings suggest that quality dialogue, thinking and talking and well-designed programmes make a difference and can impact on student outcomes (Deakin Crick 2002; Melchior, 1999). Where students were given the opportunity to discuss events in relation to their personal experiences, this aided them in connecting their world with a curriculum content that they would otherwise have seen as disparate and unrelated to their life (Faubert et al., 1996).

The implication is citizenship education can enhance students’ ability to make meaning of and connections between their personal stories and society.

Understanding and reasoning

The findings present evidence that students’ understanding and inductive and deductive reasoning were enhanced, increased, improved and developed (Deakin



Crick, 2002; Polite and Adams, 1997; Russell, 2002) in citizenship education through the use of meaningful curricula and learner-centred processes such as:

• dialogue and debate – class conferences, conversational discussions, seminars (Clare et al., 1996; Polite and Adams, 1997; Wade, 1994)

• increased participation – group and individual, presentation, research, discussion (Clare et al., 1996; Deakin Crick, 2002; Faubert et al., 1996; Polite and Adams, 1997; Russell, 2002)

• opportunities for reflection – watching theatre, journal-keeping, weekly written exercises (Day, 2002; Faubert et al., 1996; Melchior, 1999)

Students’ understanding of the process of change can influence their understanding of lesson content and their cognitive engagement (Wade, 1994) leading to a better understanding of community (Melchior, 1999). It is evident from the studies that student reasoning skills about human relationships (Beyer and Presseisen, 1995) were helped and enhanced with students better able to reason about social, moral and ethical issues of different subjects, such as science, that may impact their lives (Black and Goldowsky, 1999). The evidence suggests that students were better able to comprehend problems (Clare et al., 1996), their thinking skills went beyond levels normal for their class (Deakin Crick, 2002; Russell, 2002) and that they developed a greater reliance on problem-solving (Faubert et al., 1996).

The implication is that citizenship education can improve students’ understanding and reasoning skills.

Higher order thinking skills

There is support in the study findings by Polite and Adams (1997) that citizenship education can engage and encourage students in meta-cognitive processes and this is achieved through the processes of learner-centred pedagogy and meaningful curricula. Where teachers assisted students with developmentally appropriate questions, students’ critical thinking skills in moral or social reasoning were increased (Clare et al., 1996). Students’ conceptual thinking; ability to analyse, consider and grasp multiple ideas and abstract concepts (Deakin Crick, 2002; Tibbitts, 2001); their ability to make sense of the abstract notions of the lesson content and to reconsider their ideas in the light of new information (Wade, 1994) were all outcomes of these studies. Conceptual abstract thinking and thought provoking curricula increased students ability to link complex and moral issues (Black and Goldowsky, 1999), facilitating students to move beyond basic points and superficial understanding to explore the moral and emotional qualities of issues (Clare et al., 1996). The provision of citizenship education can result in statistically significant positive changes in formal operational thought through movement from concrete literal thinking to abstract and scientific thinking (Faubert et al., 1996) resulting in higher levels of reflection.

The implication is that the provision of citizenship education can engage and enhance students’ higher order thinking skills.

Academic achievement

Evidence from across the studies shows that citizenship education improved students’ grades and academic achievement. There was a positive impact on grades, with increased achievement, cognitive engagement and cognitive



functioning (Deakin Crick, 2002; Melchior, 1999; Polite and Adams, 1997); the development of further knowledge (Wade, 1994); and gains in cognitive abilities (Faubert et al., 1996; Garcia-Obregon et al., 2000). There was some evidence that service learning had a stronger impact on academic achievement among non-whites, along with improved school performance (Melchior, 1999).

The implication is that the provision of citizenship education can enhance students’ academic achievement.


There is evidence of increased communication skills (Black and Goldowsky, 1999; Day, 2002), which is closely related to participation and dialogue. Where students received increased opportunities to participate in dialogue and structured class discussions, they could express themselves freely and justify and explain their views (Wade, 1994). Citizenship education was found to enhance students’ abilities to develop, debate and express complex moral and social themes (Clare et al., 1996) and encouraged students to respond to each other with a deepening ability to actively listen and reciprocate to others (Clare et al., 1996; Faubert et al., 1996; Russell, 2002)

The implication is that the provision of citizenship improved students’ communication skills.

4.4.5 Discussion of findings: learning processes

Learning processes emerged as contributing to the achievement of cognitive learning outcomes.

Engagement

There is evidence from the studies that contextual knowledge, problem-based thinking and matching learning content with pedagogy seemed to result in increased student engagement, participation and action for individuals and groups (Clare et al., 1996; Day, 2002; Deakin Crick, 2002; Faubert et al., 1996; Polite and Adams, 1997; Russell, 2002). The studies present evidence that students learn in an experiential manner through ‘doing’ and through their own ‘lived’ experience and that they can apply what they have learned in this way to other situations to either express empathy or take action (Clare et al., 1996; Day, 2002). In addition, there was evidence of active programme participation having an impact on student career choice (Melchior, 1999), and that citizenship programmes can impact on student cognitive growth (Faubert et al., 1996).

The implication is that the provision of citizenship education through students’ engagement and active participation can contribute to students’ cognitive learning outcomes.

Promoting discussion

A common feature throughout the studies is the practice of increased opportunities for students to make a contribution such as in a group discussion, dialogue, conversational discourse or debate (Clare et al., 1996). These practices are highly characteristic of the processes of citizenship education and had an impact on student learning and achievement. The studies indicated that schools’ involvement in transformative interactive dialogical pedagogies and democratic



processes was not at the expense of, but complementary to, and enhancing of, academic learning and achievement. There was evidence from Russell (2002) and Clare et al. (1996) that a highly interactive and challenging dialogue process helped children refine their ideas and created an opportunity for comprehension and moral growth.

Dialogical pedagogies require quality of relationships, which are inclusive and respectful. When the learner was regarded as a whole person, with affective and volitional and cognitive needs, then teachers related differently to students (Deakin Crick, 2002). The findings show learning and teaching strategies that facilitated a conversational pedagogy, in which dialogue and discussion were the norm, because questioning and dialogue encouraged students in the processes of reflective searching for deeper meaning to issues and events (Clare et al., 1996; Day, 2002; Deakin Crick, 2002; Russell, 2002). Such strategies included weekly class conferences (Wade, 1994), student participation in formal and informal group discussions (Deakin Crick, 2002; Melchior, 1999) and Socratic dialogue (Polite and Adams, 1997).

The implication is that the provision of citizenship education promoted discussion and dialogue, which can contribute to students’ cognitive learning outcomes.

Learner-centred teaching

Some of the studies provide evidence that teachers may need to change their pedagogies and will need support in order to develop a more facilitative role and enhance their competencies of questioning, listening and summarising (Clare et al., 1996; Wade, 1994). In co-operative learning environments, where the teacher lets go of control in order to listen to the voice of students, an atmosphere of trust and safety was created that enhanced teacher/student relationships and increased participation (Clare et al., 1996; Deakin Crick, 2002; Polite and Adams, 1997; Russell, 2002). When teachers possess the technical ability skilfully to question, listen, summarise and clarify what the student has said, the students are empowered because they feel listened to and what they have said mattered and was taken seriously (Clare et al., 1996; Polite and Adams, 1997; Russell, 2002). It is an acknowledgement of the student as a person with opinions, and those opinions and contributions are of value. This can in turn lead to a boost in student self-confidence because they think and feel that they have something meaningful to offer and their stories are of value for others to hear and learn from. This is empowering for students, both cognitively and affectively (Faubert et al., 1996).

The implication is that the provision of learner-centred teaching through citizenship education can contribute to students’ cognitive learning outcomes.

Meaningful curricula

The studies provided evidence where the content of the curriculum is relevant and pertinent to students’ lives that there is increased motivation to participate (Russell, 2002; Tibbitts, 2001; Wade, 1994). The use of real-life contexts, which were highly relevant to students’ personal life experiences, and the use of different media (such as plays and theatre forum), led to students being better able to apply and make connections between events and issues and their own experiences, leading to improved learning, an improved ability to reason about their own and the experiences of others, and an improved ability to reason about the impact of developments on their lives (Beyer and Presseisen, 1995; Black and Goldowsky, 1999; Day, 2002).



The development of meaningful curricula requires a change from traditional teaching methods (Clare et al., 1996; Tibbitts, 2001; Wade, 1994). Greater interactivity, through audience participation (Black and Goldowsky, 1999) and volunteer service in the community, affects students’ ability to make meaning of the lesson content and provides a positive experience of participation (Melchior, 1999). When students are given opportunities to make contributions in a discussion or role-play scenario, they are more likely to initiate discussions (Clare et al., 1996) with students expressing a preference for seminar scenarios to the classroom environment (Polite and Adams, 1997), because they learned more than in a regular classroom environment (Wade, 1994). A facilitative pedagogy and meaningful curricula enable students to make deeper connections with lesson content and hence learn on deeper cognitive and affective levels than they might otherwise (Clare et al., 1996; Deakin Crick, 2002). The evidence suggests that more attention should be given to the use of different strategies to encourage greater student participation (Black and Goldowsky, 1999) and that highly interactive and challenging discussion approaches can be a useful tool for educators attempting to enhance comprehension (Clare et al., 1996; Deakin Crick, 2002; Wade, 1994).

The implication is that the provision of citizenship education can enhance students’ cognitive learning outcomes through relevant learning experiences and the use of different media.

Developing personally

There is evidence from across the studies that the provision of citizenship education has an impact on personal and social affective outcomes, as well as on cognitive learning outcomes in areas such as the development of self-concept (Faubert et al., 1996), increased self-confidence (Russell, 2002) and more positive behaviour (Melchior, 1999). These were positive impacts on behaviour, with students displaying less risk behaviours and more positive attitudes to society (Melchior, 1999). Students displayed greater empathic and impartial reasoning (Faubert et al., 1996; Russell, 2002) and there were increases in student motivation levels to participate and get involved, in particular when the lesson content was relevant to the students’ own experiences. Emotional salience played a significant role in conceptual change (Wade, 1994). Greater gains in self-concept development were evident with students taking greater responsibility for their own choices (Russell, 2002), developing a greater sense of autonomy, working diligently, increasing a firmer sense of self and presenting with poise and confidence (Faubert et al., 1996; Russell, 2002). Decreases in absence levels were reported with a lower number of referrals; more students were reported to be feeling better about themselves; and the programme interventions helped students get along with, and care about, others (Garcia-Obregon et al., 2000). Different learning-centred approaches allowed students to develop and apply conflict resolution skills; seminars gave students the opportunity to witness to other students’ contributions (Polite and Adams, 1997) and engage in active listening (Russell, 2002), developing increased feelings of respect for others opinions and noticing differences in peer behaviour, with more polite behaviour commented upon (Polite and Adams, 1997).

The implication is that the provision of citizenship can impact on students’ cognitive learning outcomes through helping them to develop personally.



4.5 In-depth review: quality-assurance results Data extraction for all 13 studies was carried out independently by two reviewers and, for three studies, by three members of the Review Group. EPPI-Centre staff were involved in reviewing seven of these data extractions. The main differences between reviewers related to their judgement of the nature of the study type. In each case, the reviewers moderated their findings and agreed a final judgement. One reviewer was also an author of a study and, in this case, was neither involved in the data extraction of the study nor in making any judgements concerning the study at any stage.

4.6 Nature of actual involvement of users in the review and its impact The time pressures on the completion of this review meant that a formal consultation about the implications of the findings with users was not possible. A meeting of the Review Group was convened at relatively short notice to consider the findings and to discuss the strengths and limitations of the review, as well as the implications of the findings for policy, research and practice. The consultative group comprised individuals with experience in research, in education policy formation and in practice as teachers. While this consultation was useful and produced the implications recorded in Chapter 5, it will be important to consider carefully the responses from different user groups that are asked to respond to the review after it is published.


5. Findings and implications

5. FINDINGS AND IMPLICATIONS

This chapter summarises the review process and the key findings. It discusses the strengths and limitations of this review, with its particular focus and methodology and then outlines the implications of the findings for teacher education, policy, practice and research.

5.1 Summary of principal findings The main review question is as follows:


In order to achieve all the aims of the review, it was necessary to address the following further question:


In order to report findings relevant to these questions, the studies were grouped according to their relevance to the following five categories of cognitive learning outcomes. These are subsequently grouped together under the overall heading of ‘Developing holistic approaches to achievement’

• meaning-making


• higher order thinking

• academic attainment


The five key themes that emerged as characteristic of the learning processes that made a contribution to these outcomes were as follows:

• engagement







5.1.1 Identification of studies

The search strategy identified 647 reports to which inclusion and exclusion criteria were systematically applied, of which 35 studies were included in the systematic map, including four studies from the first citizenship review. The Review Group then decided to focus only on studies that reported on cognitive outcomes and met minimum reporting quality criteria. Thus 13 studies were selected for data extraction.

5.1.2 Mapping of all included studies

The studies included in the mapping stage of the review included studies which represented a range of study types, a wide range of citizenship education across different curriculum subjects, and the full range of learning processes and learning outcomes identified by the review-specific keywords.

5.1.3 Nature of studies selected for in-depth review

Having mapped the research literature, the Review Group decided to narrow the focus of the review. The in-depth review question was as follows:

What is the impact of citizenship education on students’ cognitive outcomes?

The studies selected for in-depth review were only studies that met minimum reporting requirements and reported on the impact on cognitive learning outcomes. However, these 13 studies also addressed a range of types of citizenship education and a wide range of learning processes. A full summary of all these studies can be found in Appendix 4.2.

5.1.4 Synthesis of findings from studies in in-depth review

In summary, the findings of this review indicate that citizenship education can be applied to most areas of the curriculum through the development of learner-centred teaching, and meaningful curricula. The evidence from this review about the impact of citizenship education on student learning and achievement suggests the following:

Pedagogy appropriate for citizenship education:

• can enhance student learning and achievement;

• may be characterised by a facilitative, conversational pedagogy, where dialogue and discussion are the norm;

• can improve students’ communication skills;

• can enhance students’ academic achievement;

• can engage students to seek cognitive understanding of the meaning of their personal stories and experiences when learning about lesson content and gaining awareness of others’ situations;



• may lead to greater participation when lesson content is pertinent to student experiences;

• can enhance students’ higher order cognitive and intellectual development;

• engages and can enhance students’ meta-cognitive processes;

• allows for an increased participation and a greater interactive, and may ensure a more positive experience of participation that affects student ability to make meaning of the lesson content;

• can result in statistically significant positive changes in formal operations of movement from concrete literal thinking to abstract and scientific thinking, resulting in higher levels of reflection;

• can create a co-operative learning environment, leading to an atmosphere of trust and safety, that enhances teacher/student relationships, where teachers let go of control and listen to student voice;

• may empower students, leading to increased self-confidence, more positive self-concept and greater self-reliance;

• can engage learners as whole persons and result in teachers relating differently to students; and

• can impact on affective outcomes, as well as cognitive growth in areas such as the development of self-concept, increased self-confidence, and more positive behaviour.

In addition:

• teachers themselves may need support in order to develop expertise in facilitation and dialogue;

• learner-centred teaching and meaningful curricula can affect the motivation and cognitive engagement of students and require a change from traditional teaching methods and content;

• questioning and dialogue can encourage students in the processes of reflective searching for deeper meaning to issues and events; and

• cognitive outcomes are achieved in relation to the affective and volitional domain and not in isolation.

The findings of this review support the findings of previous reviews discussed in Chapter 1 and in particular they build upon the findings of the first review.



5.2 Strengths and limitations of this systematic review

Cognitive learning outcomes

This review both developed from and built upon the previous EPPI-Centre citizenship review, which examined the impact of citizenship education on the provision of schooling; this is directly demonstrated by four studies from the first review also being included in this review. The question for this review ‘What is the impact of citizenship education on student learning and achievement?’ had a more specific remit: the learning outcomes for students resulting from certain citizenship-related curricular and pedagogic processes. Moreover, during the implementation of the review methodology, the review question was further narrowed, partly as a matter of expediency in managing the available studies in order to engage with the technical processes of the review in the time available. The revised review focus thus became the impact on cognitive learning outcomes, albeit of a range of types of citizenship education and a wide range of learning processes.

On the one hand, a focus on cognitive learning outcomes (as defined in Appendix 2.4) might be considered to be a limitation, especially since citizenship education itself, and many studies of it, are often primarily concerned with personal and social learning (and sometimes moral and political learning). On the other hand, by deliberately placing cognitive learning in the spotlight, the review findings are able to show, by means of this clearer focus, that citizenship education pedagogies and curricular experiences can result in cognitive learning, as well as personal and social learning. In terms of evidence of cognitive learning outcomes from the studies included in the in-depth review, some measured hard cognitive learning outcomes, while others indicated qualitative softer cognitive learning outcomes, both of which types are valid in terms of the studies themselves. Indeed, by closely scrutinising and reducing the studies during the selection process, some studies, which showed personal and social learning outcomes interrelated with cognitive learning outcomes, were excluded from the in-depth review, thereby intensifying the focus on cognitive learning per se. Thus the cognitive learning outcomes from citizenship education provision and practices demonstrated by the research examined in the in-depth review is likely to be of interest to policy-makers and inspection and advisory agencies, who tend to emphasise and value cognitive learning and achievement, often at the expense of personal and social learning.

Citizenship education models and ideologies

As outlined in Chapter 1 (section 1.2), the model of citizenship education used as the framework for this review was that set out in the Crick Report (1998), based in the context of the National Curriculum of England and defining three main strands: moral and social responsibility, community involvement and political literacy. Given the location of the EPPI-Centre and the review question in an English context, this was justifiable pragmatically. However, such a model of citizenship education is by no means universal or commonly employed in other educational systems and settings. Most of the 13 research studies in the in-depth review were conducted in locations outside England and beyond the UK. This is also the case even when the in-depth review studies are contextualised in the



broader picture of the 35 research studies, which were keyworded, as these researches were also predominantly carried out in the USA (with a few in other European countries and Australia). Furthermore, the two studies reviewed in-depth, which were conducted in England, were both undertaken prior to the implementation of the Citizenship Education curriculum in England in 2002.

The review thus operated within the ideological framework of an established liberal democracy, with the presuppositions and interpretations embedded in a specific historical and educational context, and of the reviewers working within the limitations entailed by the EPPI-Centre review process. It is this English setting to which the review returns in applying the relevance of its findings drawn from a wider international canvas. However, one of the questions deserving further exploration as a result of the review is the extent to which the five identified categories of cognitive learning outcomes and the five key themes that emerged as characteristic of the learning processes that made a contribution to these outcomes are dependent on, or independent of, the ideologies and characteristic pedagogical and curricular practices of particular educational systems, schools and teachers. Moreover, given the small-scale nature of many of the studies and the numbers of schools, teachers and students involved, the generalisability of the specific findings may be limited. However, the interrelated cluster of overlapping categories and contributory processes which have emerged in the synthesis of findings, as a result of applying the EPPI-Centre review methods to the review question, gives the review’s evidence strength and pertinence. Thus, even though the current form of the newly constituted citizenship education is still emerging in England, the underlying and long-standing curricular practices, and teaching and learning processes characteristically associated with this domain (as defined in Appendix 2.4) can be seen to have relevance and application beyond the aims and learning objectives of citizenship education per se to a wider concept of education.

This grounded and rounded concept is learner-centred (involving engagement, dialogue and discussion, relating learning to experience, and holistic approaches to achievement) with demonstrable cognitive learning outcomes (such as meaning-making, understanding and reasoning, higher order thinking, communication skills and academic attainment), which in turn enable learners to develop as whole persons.

Review methodology

The methodology of the EPPI-Centre review process had several further implications for the strength and weaknesses of this review. Some of these were shared with those identified in the first citizenship education review.

Although this second review of citizenship education was more focused than the first review, it, too, was conceptually complex since it involved the definition both of citizenship education and also of learning processes and achievement outcomes. As before, the definition of citizenship education covered a broad range of educational activities, beyond that named in educational policy. By conjoining citizenship education with student learning and achievement, the review was directed towards pedagogic practices and curricular experiences with directly related achievement outcomes. In this sense, the scope of the review was very broad, with the result that, during the review procedure, it became necessary to limit the definition of achievement to cognitive learning outcomes in order to deal with the amount of available material. The effect of this procedural decision



was to limit exploration of the interrelationship between cognitive and other kinds of learning outcomes, mainly personal and social, but also moral and political, in the data-extraction process. Nevertheless, the synthesis of findings from studies in the in-depth review indicates that cognitive outcomes were not achieved in isolation from affective outcomes and personal development.

As with the first review, the procedure was to mount a systematic search to cover as many potential sources as possible, to record the process enabling a replication in principle or additions to the review if more relevant studies were to become available at a later date. However, in this case, the initial searches resulted in too much unspecific material. Following the systematic review processes and recording of the EPPI-Centre procedures facilitated the progressive focusing on core material and the identification of 13 studies central to the review question, linking pedagogy and curricular experience characteristic of citizenship education with achievement outcomes. A strength of the process was the independent double-checking of judgements of inclusion/exclusion criteria, keyword applications and data extraction by reviewers, and the further sample check by EPPI-Centre colleagues. On the other hand, the process can be seen as overly technicist, following a medical or economic model of data reporting, validity and reliability, whose presumed objectivity does not always fit so well with educational studies dealing with complex human interactions, learning processes and environments, and often subjective judgements about evidence of student learning.

Studies included in the review had not necessarily been undertaken to address issues close to the review question. The assessment of weight of evidence in relation to the review question enabled overall judgements to be made about the contribution of studies to the review. In this review, however, the papers were of varying quality; of those included in the in-depth analysis, a few studies were rated as having high trustworthiness and/or appropriateness or relevance, but only one was considered to have high review-specific weight of evidence and trustworthiness.

Another issue for this review was the timescale of the review. Initially it had been intended that the review would take until November 2004. However, the Review Group was requested by the EPPI-Centre to carry out and complete the reporting of the review in a shorter timescale by the end of July 2004 as the current contract between the DfES and the EPPI-Centre was due to end on the 31 January 2005. This put pressure on the tight part-time staffing arrangements of the project, given the lack of additional funding. It also had a negative effect on the consultation process with the Citizenship Education Research Strategy Group, comprising experts drawn from policy development, research and practitioners at various levels of the education system, since the foreshortened project deadline meant holding a meeting of the group at relatively short notice at the end of the traditional summer holiday period/beginning of the school year, which members were unable to attend. Although written comments were invited, the timescale in effect reduced the common sharing in the consultative process. As a result, the Review Group alone was responsible for identifying the implications of the findings of this review for policy, practice and research.



5.3 Implications

5.3.1 Policy

If the findings of this review are taken seriously, they have clear implications for a radical review of the system and structure of schooling so as to incorporate citizenship education strategies to reconceptualise pedagogy as learner-centred and to develop approaches to achievement, which are holistic.

Professional learning

In particular, there are significant implications of the review for teacher professional education. This review, like the first, clearly has implications for the professional learning and practices of all teachers, at all phases and in all subjects, not only Citizenship Education, who are concerned to facilitate meaningful learning outcomes.

Much of teacher training is functional and, in practice, teachers become concerned with issues of control and teaching to the syllabus and test and examination requirements. The findings of this research review suggest that, in initial teacher training (ITT) and continuous professional development (CPD), there is a need for teachers to be supported to develop a richer, deeper, broader process-oriented pedagogy. This involves having a new understanding and vision of learning and achievement seen from a more holistic perspective, where different kinds or categories of learning are viewed as complementary not separate. A citizenship pedagogy, based on the key themes characteristic of learning processes identified in the review, will have at its core communication, facilitating and enabling, dialogue and discussion, and encouragement to engage with learning, relating learning to experience. This more conversational and negotiated style of teaching and learning involves mutually respectful teacher-student relationships where traditional authoritarian patterns of control are no longer appropriate. The wider process of learning includes learning the concepts and practices central to citizenship education: discussion, having a voice, learning to listen, respect, equality, rights, responsibility, fairness and authority. For teachers to be able to adopt such processes and practices as central to their professional behaviour in the classroom, school and in community-based learning experiences, they need to be supported by school leadership and a congruent and pervasive ethos of learning.

In particular, teachers will need support to develop the specific pedagogical skills in facilitating the learning and teaching processes, which the research studies indicate enable cognitive learning from citizenship education. In both ITT and CPD, teachers will, for example, need to learn how to problematise both content and pedagogy, to develop skills of discussion, to make connections with real life experiences, to draw out meaning, to enable students to deal with cognitive dissonance and conceptual conflict, and engage in higher order thinking, so there is both personal and cognitive development for teacher and students. This can also result in citizenship engagement and action, if school and community structures enable participative and democratic processes. Teachers need support to develop the confidence to model the six key themes identified as characteristic of learning processes in this review.



5.3.2 Practice

A citizenship pedagogy has implications for the curriculum structure and curriculum delivery, as well as for evaluating and assessing learning.

As citizenship education practices and processes, which promote student learning and achievement cut across the curriculum and whole-school provision, they suggest the need for curricular flexibility, with more opportunities to vary timetables to allow for different groupings of learners and provision of accommodation which facilitates more interactive and conversational learning styles.

Learning through citizenship education requires a wider range of learning environments, in and beyond school. As the table of 35 selected studies shows (Table 3.1), as many of the research studies were keyworded as not focusing on curriculum issues as were those having a curricular citizenship focus. Traditionally, and in accordance with the community involvement strand of the Crick Report, the local community beyond the school has often been seen as offering opportunities for citizenship learning, with the development of skills, emotions, knowledge and understanding. The research studies in this review demonstrate the ability of active service learning, drama, role-play, games and interdisciplinary approaches to contribute to cognitive as well as personal and social learning outcomes. The community for citizenship learning is not just that within the school boundaries, but in participation with the communities beyond.

Just as there is a need to link learning environments within the school and beyond in terms of processes and outcomes, so schools need to become more conscious of linking the formal and informal contexts of learning within the school. This involves teachers modelling citizenship pedagogy as learning exemplars in classrooms, corridors and other school structures. Cross-cutting pedagogies, such as team planning and work, mentoring, preparation and debriefing, and reflection are important to enable learning from experience, facilitate understanding and reasoning, make connections and meaning, and transfer learning to other areas.

Transformative pedagogies require learner-centred, differentiated, meaningful reporting of learning as part of the learning through assessment experience. Such formative assessments can be used for diagnostic purposes. A holistic approach to achievement means assessing a wider range of competencies and recording them in diverse ways to produce a personal learning portfolio. This can promote effective use of self-evaluation by students and diagnostic evaluation by teachers. During learning and assessment processes, teachers can be encouraged, with students, to develop indicators of effective citizenship in learning outcomes – attendance, helpful and considerate behaviour in/outside school, developing participatory skills, demonstrating respect for diversity and difference, by honouring other traditions and cultural narratives, as well as developing awareness of shared similarities, common needs and values. Such outcomes were reported in the studies reviewed.

5.3.3 Research

Although the review found many studies which addressed some aspect of citizenship education with student learning and achievement, this most often involved personal and social learning outcomes, rather than cognitive learning per se. Much of the research was at the level of small case-studies or single school



studies, which has implications for generalisability. The quality of studies included in the in-depth review varied significantly. Of the 13 studies, two were judged to have low trustworthiness, six were judged to have medium trustworthiness and five were judged to have high trustworthiness. Undertaking and making judgements about citizenship education research is challenging and interpretative since it involves processes, phenomena and qualities, which are not always or often easily and objectively evaluated or measured. Reporting of research often requires more transparency, problematising of the issues and their interpretation, and offering theoretical and explanatory rationales for researcher judgements.

As a result of reviewing available research related to the research question ‘What is the impact of citizenship education on student learning and achievement?’, it seems there is a need for more interdisciplinary research, which employs mixed methods, quantitative and qualitative, and which involves in-depth study of several schools. Studies are needed of the effects of different citizenship models and pedagogies on cognitive learning outcomes, and into ways to link such learning more carefully and systematically to complementary personal and social learning. Since citizenship education is about lifelong learning and practices, more research is also needed into the impact of family and community-based learning and its interrelation with school-based learning in citizenship education.


6. References

6. REFERENCES

6.1 Studies included in map and synthesis *Those studies included in both the map and the synthesis are marked with an asterisk. Annenberg Rural Challenge (1999) The Rural Challenge: Rural Schools and Communities Working as Partners for the Future of Rural America. Granby, CO: Annenberg Rural Challenge. *Beyer FS, Presseisen BZ (1995) Facing History and Ourselves: Initial Evaluation of an Inner-City Middle School Implementation. Philadelphia, PA: Research for Better Schools, Inc. Bickmore K (1993) Learning inclusion/inclusion in learning: citizenship education for a pluralistic society. Theory and Research in Social Education 21: 341–384. *Black DR, Goldowsky A (1999) Science theater as an interpretive technique in a science museum. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching. Boston, MA: March 28–31. Carter C, Osler A (2000) Human rights, identities and conflict management: a study of school culture as experienced through classroom relationships. Cambridge Journal of Education 30: 335–356. Carter CC (1999) Conflict mediation at school: peace through avoidance? Paper presented at the Annual Meeting of the American Educational Research Association. Montreal, Canada: April 19–23. *Clare L, Gallimore R, Patthey-Chavez G (1996) Using moral dilemmas in children’s literature as a vehicle for moral education and teaching reading comprehension. Journal of Moral Education 25: 325–341. *Day L (2002) ‘Putting yourself in other people’s shoes’: The use of forum theatre to explore refugee and homeless issues in schools. Journal of Moral Education 31: 21–34. *Deakin Crick R (2002) Transforming Visions, Managing Values in Schools. London: Middlesex University Press. *Faubert M, Locke DC, Sprinthall NA, Howland WH (1996) Promoting cognitive and ego development of African-American rural youth: a program of deliberate psychological education. Journal of Adolescence 19: 533–543. *Garcia-Obregon Z, Trevino J, Uribe-Moreno S, Zuniga S (2000) The Effectiveness of a School Based Service-Learning Program ‘Community Connection’ at a South Texas Middle School. Texas: South Texas Research and Development Center.


6. References

Hecht D, Fusco DR (1996) Student perceptions of the opportunities available from participating in service learning. Paper presented at the Annual Meeting of the American Educational Research Association. New York, NY: April 8–12. Horwood B, Patterson B (1995) Community involvement in education. In: Horwood, B. (Ed.), Experience and the Curriculum. Boulder, CO: Association for Experiential Education, pages 51–67. Johns S, Kilpatrick S, Falk I, Mulford B (2000) Leadership from Within: Rural community revitalisation and the school-community partnership. Launceston, Australia: Tasmania University, Center for Research and Learning in Regional Australia. Johnson DW, Johnson R, Dudley B, Ward M, Magnuson D (1995) The impact of peer mediation training on the management of school and home conflicts. American Educational Research Journal 32: 829–844. Johnson DW, Johnson R, Mitchell J, Cotten B, Harris D, Louison S (1996) Effectiveness of conflict managers in an inner-city elementary school. Journal of Educational Research 89: 280–285. Jordan D, Le Metais J (1997) Social skilling through cooperative learning. Educational Research 39: 3–21. Kim S, Parks S, Beckerman M (1996) Effects of participatory learning programs in middle and high school civic education. Social Studies 87:171–176. *Laconte MA, Shaw D, Dunn I (1993) The effects of a rational-emotive affective education program for high-risk middle school students. Psychology in the Schools 30: 274–281. *Melchior A (1999) Summary Report: National evaluation of Learn and Serve America. Waltham MA: Center for Human Resources, Brandeis University. Available from: http://www.learnandserve.org/pdf/research/lsreport.pdf Morais A, Rocha C (1999) Development of social competences in the primary school: study of specific pedagogic practices. British Educational Research Journal 26: 91–119. Nelson A (2003) Peer mentoring: a citizenship entitlement at Tanfield school. Pastoral Care in Education 21: 34–41. Obenchain KM (1998) Preparing active and informed citizens: a qualitative study of two elementary classrooms. Paper presented at the Annual Meeting of the National Council for the Social Studies. Anaheim, CA: November 18–19. Ogden L (2000) Collaborative tasks, collaborative children: an analysis of reciprocity during peer interaction at Key Stage 1. British Educational Research Journal 26: 211–226. Pate PE, Thompson KF, Keyes M (2001) Students, standards and exploration: a responsive, relevant and engaging curriculum. Middle School Journal 33: 35–40. Pipkin R, Yates D (1992) Multicultural education: a middle school’s approach to preparing tomorrow’s leaders. ERS Spectrum 10: 37–40.


http://www.learnandserve.org/pdf/research/lsreport.pdf

6. References

Pitiyanuwat S, Sujiva S (2001) Civics education in Thailand: three case schools. International Journal of Educational Research 35: 93–108. *Polite VC, Adams AH (1997) Critical thinking and values clarification through Socratic seminars. Urban Education 32: 256–278. *Russell J (2002) Moral consciousness in a community of inquiry. Journal of Moral Education 31: 142–153. Scales PC (1999) Increasing service-learning’s impact on middle school students. Middle School Journal 30: 40–44. Schmuck P, Schmuck R (1990) Democratic participation in small-town schools. Educational Researcher 19: 14–19. Sewell AM, St George AM (1997) Self-efficacy in the classroom: developing the skills of citizenship. Children’s Social and Economics Education 2: 136–144. Stevahn L, Johnson DW, Johnson RT, Schultz R (2002) Effects of conflict resolution training integrated into a high school social studies curriculum. Journal of School Psychology 142: 305–331. *Tibbitts F (2001) Prospects for civics education in transitional democracies: results of an impact study in Romanian classrooms. Intercultural Education 12: 27–40. *Wade RC (1994) Conceptual change in elementary social studies: a case study of fourth graders’ understanding of human rights. Theory and Research in Social Education 22: 74–95.

6.2 Other references used in the text of the report Adey PA (2001) The categorisation and assessment of ‘thinking skills’. Paper presented at the EARLI conference. Fribourg, Switzerland: 28 August – 1 September. Ames C (1992) Classrooms: goals, structures, and student motivation. Journal of Educational Psychology 84: 261–271. Ames C, Ames R (eds) (1984) Research on Motivation in Education: The classroom milieu, Vol 2. New York: Academic Press. Bloomer M (2001) Young lives, learning and transformation: some theoretical considerations. The Oxford Review of Education 27: 429–447. Brown A, Palinscar AS (1982) Inducing strategic learning from text by means of informed self-control thinking. Topics in Learning Disabilities 2: 1–17. Brophy J (1987) Socializing students’ motivation to learn. In: Maehr ML, Kleiber DA (eds) Advances in Motivation and Achievement, Volume 5. Greenwich, CT: JAI Press, pages 181–210.


6. References

Clark E (1997) Designing and Implementing an Integrated Curriculum. Brandon VT: Holistic Education Press. Crick B (1998) Education for Citizenship and the Teaching of Democracy in Schools: Final report of the Advisory Group on Citizenship. London: Qualifications and Curriculum Authority. Cuban L (1984) Policy and research dilemmas in teaching of reasoning: unplanned designs. Review of Educational Research 54: 655–681. Deakin Crick R, Coates M, Taylor M, Ritchie S (2004) A systematic review of the impact of citizenship education on the provision of schooling. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education. De Bono E (1983) The direct teaching of thinking as a skill. Phi Delta Kappa 64: 703–708. Ennis RH (1993) Critical thinking assessment. Theory into Practice 32: 179–186. Ennis RH (1996) Critical thinking dispositions: Their nature and assessability. Informal Logic 18: 165–182. Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) (2002a) Core Keywording Strategy: Data collection for a register of educational research. Version 0.9.7. London: EPPI-Centre, Social Science Research Unit, Institute of Education. Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) (2003b) Guidelines for Extracting Data and Quality Assessing Primary Studies in Educational Research. Version 0.9.5. London: EPPI-Centre, Social Science Research Unit, Institute of Education. Fogarty R (1992) Beyond test scores: tracking significant outcomes. Cogitare 6: 2. Fogarty R, McTighe J (1993) Educating teachers for higher order thinking: the three-story intellect. Theory into Practice 32: 161–169. Freire P (1972) Pedagogy of the Oppressed. Harmondsworth: Penguin. Gardner H (1993) Frames of mind: the theory of multiple intelligences. London: Fontana. Gearon L (2003) Citizenship Education: A Professional User Review of Research. Nottinghamshire: British Educational Research Association. Gilligan C (1982) In a Different Voice: Psychological Theory and Women’s Development. Cambridge MA: Harvard University Press. Gray A (1999) Time off Pays off: How Reduction in Working Time can Create Jobs and Promote Lifelong Learning. London: Fabian Society.


6. References

Huitt W (1998) Critical thinking: an overview. In: Educational Psychology Interactive. Valdosta, GA: Valdosta State University. Available from: http://chiron.valdosta.edu/whuitt/col/cogsys/critthnk.html Johnston C (1996) Unlocking the will to learn. Thousand Oaks, California: Corwin Press. Johnson D, Johnson R (1985) Motivational processes in cooperative, competitive, and individualistic learning situations. In: Ames C, Ames R (eds) Research on Motivation in Education: The Classroom Milieu, Volume 2. New York, NY: Academic Press, pages 249–286. Kelly G (1955) The Psychology of Personal Constructs. Volumes 1 and 2. New York: Norton. Kerr D (1999a) Citizenship Education: An International Comparison. London: QCA/NFER. Kerr D (1999b) Re-examining Citizenship Education: The Case of England. Slough: NFER. Kerr D, Lines A, Blenkinsop S, Schagen I (2001) Citizenship and Education at Age 14: A Summary of the International Findings and Preliminary Results for England. Slough: NFER. Kerr D (2002) An international review of citizenship in the curriculum: the IEA national case studies and the INCA archive. In: Torney-Purta J, Steiner-Khamsi G, Schwille J (eds) New Paradigms and Recurring Paradoxes in Education for Citizenship. Amsterdam: Elsevier Press, pages 291–334. Kerr D (2003) Citizenship education in international perspective. In: Gearon L (ed.) Learning to Teach Citizenship in the Secondary School. London: Routledge, pages 5–27. Lawson AE (1993) At what levels of education is the teaching of thinking effective? Theory into Practice 32: 170–178. Lewis A, Smith D (1993) Defining higher order thinking. Theory into Practice 32: 131–137. Loevinger J (1976) Ego Development: Conceptions and Theories. San Francisco: Jossey-Bass. McCombs BL, Whisler JS (1997) The Learner Centered Classroom and School. San Francisco: Jossey-Bass. McGettrick B (2002) Emerging Conceptions of Scholarship, Service and Teaching. Toronto: Canadian Society for the Study of Education (CSSE). Marzano R, Hutchins R (1985) Thinking Skills: A Conceptual Framework. A special issue of Noteworthy. Aurora, CO: Mid-continent Regional Educational Laboratory.


http://chiron.valdosta.edu/whuitt/col/cogsys/critthnk.html

6. References

Newman FM (1990) Higher order thinking in teaching social studies: a rationale for the assessment of classroom thoughtfulness. Journal of Curriculum Studies 22: 41–56. Renner JW, Stafford DG, Lawson AE, McKinnon JW, Friot FE, Kellog DH (1976) Research, Teaching and Learning with the Piaget Model. Norman, OK: University of Oklahoma. Roth KJ (1990) Developing meaningful conceptual understanding in science. In: Jones B, Idol L (eds) Dimensions of thinking and cognitive instruction. Hilldale, NJ: Erlbaum, pages 139–175. Seligman MEP, Csikszentmihalyi M (2000) Positive psychology: an introduction. American Psychologist 55: 5–14. Smith J, Spurling A (1999) Lifelong Learning: Riding the Tiger. London: Cassell. Strein W (1988) Classroom-based elementary school affective education programs: a critical review. Psychology in the Schools 25: 288–296. Teacher Training Agency (TTA) (2004) Qualified to Teach. Handbook of Guidance. London: TTA. Torney-Purta J, Lehmann R, Oswald H, Schulz W (2001) Citizenship and Education in Twenty-Eight Countries: Civic Knowledge and Engagement at Age Fourteen (Executive Summary). Amsterdam: International Association for the Evaluation of Educational Achievement. Winston J (1999) Theorising drama as moral education. Journal of Moral Education 28: 459–471. Vernon A (1989) Thinking, Feeling, Behaving: An Emotional Education Curriculum for Adolescents. Champaign, IL: Research Press. Website addresses within the text

American Psychological Organisation http://www.apa.org/ed/lcp.html Teacher Training Agency Citized Project http://www.citized.info/


http://www.apa.org/ed/lcp.html


Appendix 1.1: Advisory Group membership

Appendix 1.1: Advisory Group membership

Professor James Arthur, Christchurch University College, Canterbury Ms Sheila Bloom, Institute for Global Ethics, UK Trust Mr Tony Breslin, Citizenship Foundation Dr Peter Brett, St. Martin’s University College, Lancaster *Dr Marilyn Tew, University of Bristol (Reviewer) Dr Ian Davies, Department of Educational Studies, University of York Professor Jon Davison, London Metropolitan University Mr Richard Davison, Head of PSHE and Citizenship, St George Community

School, Bristol *Dr Ruth Deakin Crick, University of Bristol (Review Co-ordinator) Dr Liam Gearon, Centre for Research in Human Rights, University of Surrey Mr Malcolm Lewis, Graduate School of Education, University of Bristol Ms Janet Palmer, Manchester Metropolitan University Mr Mark de Pulford, Lord Chancellor’s Office Dr Sue Jackson, Birkbeck College, University of London Mr David Kerr, NFER Ms Sandie Llewellin, The Post 16 Citizenship Group Professor Bart McGettrick, University of Glasgow (Chair) Anne Raven, Schools Advisor PSHE and Citizenship, Oxford *Ms Susan Ritchie, University of East London (Reviewer) Mr Don Rowe, Citizenship Foundation *Dr Monica Taylor, Journal of Moral Education and Institute of Education,

University of London (Reviewer) Ms Jan Urban Smith, Schools Advisor PSHE and Citizenship, Gloucester EPPI-Centre staff

Ms Rebecca Rees, Education Research Officer Ms Zoe Garrett, Education Research Officer Reviewers

*Elspeth Samuel *Keren Durant

*Asterisk indicates Review authors.


Appendix 2.1: Inclusion and exclusion criteria


Studies were included if they met all the inclusion criteria.

(i) Type of citizenship education

The study reports, as a specific focus, on an aspect of citizenship education that is a planned curricular intervention by schools that directly aims to address education for citizenship. This could be delivered as a discrete part of a school timetable or it could be cross-curricular or extra-curricular. As noted in the background to this review, citizenship education includes a range of important aspects of schooling, including personal development and spiritual, moral, social and cultural development, political literacy, community involvement and other terms in use, such as service learning or character education.

(ii) Type of outcome

The study reports on the process of learning and/or student achievement. For the purpose of this review, the former consists of learning dispositions, higher- order thinking skills, meta-cognition, inter- and intra-personal awareness of learning, collaborative learning and motivation for learning. The latter consists of academic, personal or social achievement as assessed by teachers, examinations or self-assessment by students.

(iii) Type of study and study design

The study is empirical. It might be a descriptive study that looked at any aspect of citizenship education (descriptive study) as implemented in the context of student learning or achievement; or a study which explored the relationship between an aspect of citizenship education and student learning and achievement; or a study that was an evaluation of a naturally occurring or researcher-manipulated intervention which was a citizenship education intervention (evaluation) which addressed student learning or achievement. Reviews and meta-analyses were not considered appropriate for data extraction but formed a useful background reference for the review. A study that described or theorised about an aspect of citizenship education and its relationship to learning and achievement, without being based directly on empirical research, was not included. An empirical study was defined as one that reports on research and provides information about a specific research question, data collection and data analysis processes, and the findings that emerge from these.

(iv) Setting and population

The study reports on students in school or pre-school, between the ages of 4 and 18. Studies dealing only with students outside this age range, in further or higher education or adult education, were excluded.

(v) Date of research

Since the focus of the review relates to the impact of citizenship education, relevant studies were considered to be those reported in 1988 or after – that is, since the beginning of the period of major policy changes with regard to citizenship and preparation for adult life in the UK, and many other countries.



(vi) Language of report

Studies were included only if they were written in English.

The following labels were attached to each excluded record to identify the reason for exclusion:

a1 not citizenship education a2 not learning a3 not achievement b not schools c not conducted post 1988 d not English language e not empirical research


Appendix 2.2: Search strategy for electronic databases


List of ERIC search terms which included selected sub-terms

Citizenship Education

S1 Human rights/ or community involvement/ or service learning/ or emotional intelligence/ or cultural education/ or bullying/ or ethical instruction/ or humanistic education/ or values clarification/ or values education/ or peer mediation/ or peer modelling/ or social education/ or social efficacy/or spiritual/ or spiritual development/ or spiritual value/ or spiritual well being/ or moral education/ or personal and social and moral education (PSME)/ or moral reasoning/ or moral reflection/ or citizenship participation/ or citizenship responsibility/ or civics/ or ethical instruction/ or values education/ or citizenship education

Schools 4-18

S2 boarding schools/ or British Infant Schools/ or community schools/ or consolidated schools/ or day schools/ or disadvantaged schools/ or elementary schools/ or folk schools/ or free schools/ or freedom schools/ or laboratory schools/ or magnet schools/ or middle schools/ or military schools/ or neighborhood schools/ or nursery schools/ or open plan schools/ or private schools/ or public schools/ or racially balanced schools/ or regional schools/ or rural schools/ or secondary schools/ or single sex schools/ or slum schools/ or small schools/ or special schools/ or state schools/ or suburban schools/ or urban schools

Learning and achievement

S3 Metacognition/ critical thinking/ or problem-solving/ or thinking skills / or learning strategies/ or cooperative learning/ or discovery learning/ or discrimination learning/ or experiential learning/ or lifelong learning/ or mastery learning/ or multi-sensory learning/ or observational learning/ or learning experience/ or learning modalities/ or learning modules/ or learning motivation/ or learning problems/ or learning processes/ or learning readiness/ or learning strategies/ or learning/ or academic achievement/ black achievement/ or high achievement/ or knowledge level/ or low achievement/ or overachievement/ or underachievement/ or achievement tests/ or aptitude/ or aspiration/ or competence/ or failure/ or fear of failure/ or qualifications/ or recognition (achievement)/ or self efficacy/ or standards/ or success/ or talent/ or achievement/ or appraisal/ or educational assessment/ or formative evaluation/ or holistic evaluation/ or informal assessment/ or peer evaluation/ or self evaluation (individuals)/ or standards



Appendix Table 2.1: History of database searching

All databases were searched via Dialog.

Database searched

Date and person Search strategy Number of hits

Imported into which database

ERIC – CIJE and RIE 1990–2003

01.04 RDC and MWT

478 Imported using ERIC dialog@Btinf filter into Endnote Database

ERIC – CIJE and RIE 1984–1989

16.01.04 RDC and MWT

80 Cut and pasted into Endnote Database

Australian Educational Index (AEI) 1976–2003

01.04 RDC and MWT

11 Cut and pasted into Endnote Database

British Educational Index (BEI) 1976–2003

01.04 RDC and MWT

0

Psychinfo 1988–2004

02.04 MWT 6 Imported into Endnote Database

ICI 1988–2004 02.04 MWT 2 Imported into Endnote Database

Education Research Abstracts 1988–2004

02.04 MWT 54 Imported into Endnote Database

BeCaL 02.04 MWT 0 REGARD 02.04 MWT 0 Removal of

duplicates gave a total of 582

Information Gateways

03.04

Handsearching of journals as listed in protocol and Appendix 2.3

02 and 03.04 MWT

65

Cut and pasted into an Endnote Database


mailto:dialog@BTinf

Appendix 2.3: Journals handsearched


Journal title Dates searched

Number of articles

American Educational Research Journal February 2004

0

American Journal of Education 1995–2001 0 American Journal of Educational Research 1997–2003 0 American Journal of Evaluation 1997–2003 0 American Research Journal 1997–2003 0 Assessment in Education 1997–2003 1 Australian Journal of Education Not available British Educational Research Journal 2000–2003 13 British Journal of Sociology of Education 1999–2003 6 British Journal of Educational Psychology 2000–2003 6 British Journal of Educational Studies 1997–2003 0 British Journal of Curriculum and Assessment 2003 0 British Journal of Educational Technology 1997–2003 0 Cambridge Journal of Education 2000–2003 2 Citizenship Studies 1997–1999 0 Cognition and Instruction 1999–2003 0 Contemporary Educational Psychology 1997–2003 0 Curriculum Journal 2000–2003 0 Developmental Review Not available Education Review 1999–2003 0 Education Technology Research and Development Not available Educational Evaluation and Policy Analysis Not available Educational Researcher 1999–2003 0 Educational Psychologist 2000–2003 0 Educational Studies 1999–2003 0 Educational Philosophy and Theory 2000–2003 0 Elementary School Journal 1996–2003 0 European Educational Research Journal 2002–2004 0 European Journal of Education 2000–2003 1 European Journal of Teacher Education 2000–2003 1 Evaluation 1999–2003 0 Forum Not available Gender and Education 1995–2003 0 Globalisation, Societies and Education 2003 0 Harvard Educational Review 1993–2002 0 Instructional Science 1998–2001 0 Intercultural Education 2000–2003 2 International Journal of Educational Research 1988–2002 1 International Review of Education 1997–2001 0 International Studies of Sociology of Education 1991–2003 0 International Journal of Children’s Spirituality 2000–2003 0 International Journal of Inclusive Education 1999–2003 0 International Journal of Leadership in Education 1999–2003 1 Journal of Beliefs and Values 2000–2003 0 Journal of Curriculum Studies 1997–2003 0 Journal of Education for Teaching: International 1995–2003 0 Journal of Education Policy 1999–2003 0 Journal of Educational Research 2000–2003 16 Journal of Intercultural Studies 2000–2003 0 Journal of Moral Education 1999–2003 1



Journal of Research and Development in Education Not available Learning and Research Not available Multicultural Education Abstracts 1994–2003 1 Oxford Review of Education 1999–2003 0 Pastoral Care in Education 1997–2003 3 Phi Delta Kappan 1991–2003 3 Research and Pedagogy Not available Research Papers in Education 2000–2003 1 Review of Education, Pedagogy and Cultural Studies

2000–2003 0

Review of Educational Research 1996–2002 0 Review of Research in Education Not available Race, Ethnicity and Education 1998–2003 0 Reflective Practice 2000–2003 0 Religious Education 2001–2003 0 Studies in Educational Evaluation 1988–2003 0 School Leadership and Management 1997–2003 1 Sex Education 2001–2003 0 Sociology of Education 1988–2003 0 Teachers and Teaching: Theory and Practice 2000–2003 0 Teachers College Record 1999–2003 0 Westminster Studies in Education 2000–2003 0 * Five studies included in stage one as ‘journal handsearching’ in the identification of potential studies, were not recorded in this table.


Appendix 2.4: EPPI-Centre Keyword sheet including review-specific keywords

Appendix 2.4: EPPI-Centre keyword sheet, including review-specific keywords V0.9.7 Bibliographic details and/or unique identifier

A1. Identification of report Citation Contact Handsearch Unknown Electronic database (Please specify.) ................................. A2. Status Published In press Unpublished A3. Linked reports Is this report linked to one or more other reports in such a way that they also report the same study? Not linked Linked (Please provide bibliographical details and/or unique identifier.) ............................................................. ............................................................. ............................................................. ............................................................. A4. Language (Please specify.) ............................................................. A5. In which country/countries was the study carried out? (Please specify.) ............................................................. ............................................................. .............................................................

A6. What is/are the topic focus/foci of the study? Assessment Classroom management Curriculum* Equal opportunities Methodology Organisation and management Policy Teacher careers Teaching and learning Other (Please specify.)......................... A7. Curriculum Art Business studies Citizenship Cross-curricular Design and technology Environment General Geography Hidden History ICT Literacy – first language Literacy further languages Literature Maths Music PSE Physical education Religious education Science Vocational Other (Please specify.).........................

A8. Programme name (Please specify.) ................................................................. A9. What is/are the population focus/foci of the study? Learners Senior management Teaching staff Non-teaching staff Other education practitioners Government Local education authority officers Parents Governors Other (Please specify.)............................ A10. Age of learners (years) 0–4 5–10 11–16 17–20 21 and over A11. Sex of learners Female only Male only Mixed sex

A12. What is/are the educational setting(s) of the study? Community centre Correctional institution Government department Higher education institution Home Independent school Local education authority Nursery school Post-compulsory education institution Primary school Pupil referral unit Residential school Secondary school Special needs school Workplace Other educational setting (Please specify) ..................................................... A13. Which type(s) of study does this report describe? A. Description B. Exploration of relationships C. Evaluation

a. naturally-occurring b. researcher-manipulated

D. Development of methodology E. Review

a. Systematic review b. Other review



Review-specific keyword definitions

Learning processes Definitions

Creative thinking

This involves relating together principles, ideas, information and entities in new and original ways to generate new entities or ideas.

Critical thinking

This involves the evaluation of arguments or propositions in relation to evidence, reasoning, drawing conclusions.

Meta-cognition

Meta-cognition refers to higher order thinking that involves active control over the cognitive processes engaged in learning. Activities such as planning how to approach a given learning task, monitoring comprehension, and evaluating progress toward the completion of a task are meta-cognitive in nature.

Experiential learning This is learning that comes from reflection on direct experience. Meaning-making

This is a person’s capacity to make personal meaning from information by integrating different kinds of knowledge. It is the ability to link information gained from different learning arenas and throughout a person’s personal history which provides an integration of different kinds of knowing – at personal, group, societal and global levels and across life contexts (home, school and community).

Inter-personal awareness

This is concerned with the capacity to understand the intentions, motivations and desires of other people. It allows people to work effectively with others. Educators, salespeople, religious and political leaders, and counsellors all need a well-developed interpersonal intelligence, including empathy.

Intra-personal awareness

This entails the capacity to understand oneself, to appreciate one’s feelings, fears and motivations. In Gardner’s (1993) view, it involves having an effective working model of ourselves, and being able to use such information to regulate our lives.


This category includes the range of communication skills: for example, listening, speaking, writing, persuading and influencing etc.

Collaboration

This is the capacity to work with other people in a cooperative way, drawing and building on their ideas and willingly contributing their own.

Problem-solving/decision-making

This involves the whole process of arriving at a decision from generating a range of ideas, screening the ideas to select the most profitable one(s) and employing a mechanism for choosing a limited number for implementation.

Values awareness

This concerns increasing awareness of the values that are held and the implications those values have for life and learning.

Achievement Definitions

Cognitive outcomes

Assessment evidence of achievement in higher order creative and critical thinking skills, problem-solving, analysis

Personal outcomes

Assessment evidence of achievement in personal development, including values, attitudes, dispositions

Social outcomes

Assessment evidence of achievement in social development, including empathy, engagement in community, service learning, collaboration and team work

Moral and political outcomes

Assessment evidence of achievement in moral and political development, including ability to hold a particular point of view and to act on it



Citizenship Education Educational programmes which are designed

to: Moral and social responsibility Develop in learners’ moral and social attitudes, values,

beliefs and behaviours. Community involvement Engage learners in learning and service in the wider

community or the school community. Political literacy Equip learners with the knowledge, skills, values,

attitudes and know how to engage in public life. Spiritual moral social and cultural development

Develop in learners any aspect of personal development which is not measured as a cognitive learning outcome.

Education for diversity Nurture in learners an understanding of difference between groups and cultures and a capacity to engage positively with groups and cultures ‘different in some way from me’. Includes race, disability, gender, religion, ethnicity, sexual orientation.

Character education Contribute to the formation of a person’s values, virtues, character and behaviour, which is beneficial to self, others and society.

Emotional and social literacy Develop the capacity to understand one’s own emotions, others’ emotions and to use that knowledge effectively.

Values education Nurture learners in an understanding of and a personal engagement with values.

Service learning Engage learners in learning which is constructed as service in the wider community or the school community.

Conflict resolution Enable learners to understand and resolve personal and communal conflicts.

Peer mediation Enable learners to support peers in resolving conflicts. Human rights education Lead to an understanding of human rights and an

engagement with the values of human rights legislation.


Appendix 3.1: Details of studies included in the systematic map


Study Country Topic focus Curriculum Population focus Study type Educational

setting Identification Citizenship education Learning Achievement

Annenberg Rural Challenge (1999)

USA Curriculum

Citizenship

Other topic focus:Community involvement

Learners Evaluation: Naturally occurring

Primary school Secondary school

Electronic database: ERIC and BEI

Moral and social responsibility Community involvement Spiritual moral social and cultural development Education for diversity Character education Values education Service learning Conflict resolution Peer mediation Human rights education

Creative thinking Meaning-making Inter-personal awareness Intra-personal awareness Communication skills Collaboration Problem-solving/decision-making Values awareness

Cognitive outcomes Personal outcomes Social outcomes Moral and political outcomes

Beyer and Presseisen (1995)

USA Curriculum

Cross-curricular: integrated the programme through maths, art, history, French, science and music

Teaching and learning

History historical knowledge of the holocaust Other curriculum

Learners Evaluation: Researcher-manipulated

Secondary school: American middle school

Handsearch

Moral and social responsibility Moral and social responsibility Spiritual moral social and cultural development Spiritual moral social and cultural development

Critical thinking Meaning-making Meaning-making Inter-personal awareness Values awareness

Cognitive outcomes Social outcomes Moral and political outcomes





Values education Human rights education

Bickmore (1993) USA Curriculum Teaching and learning

Citizenship World / Social Studies Hidden Inclusion in learning

Learners Teaching staff

Evaluation: Naturally occurring

Secondary school Electronic database: ERIC and BEI

Moral and social responsibility Political Literacy Education for diversity Emotional and social literacy Values education

Inter-personal awareness Intra-personal awareness Values awareness

Personal outcomes Social outcomes Moral and political outcomes

Black and Goldowsky (1999)

USA Curriculum Teaching and learning

Citizenship Science


Secondary school Other educational setting: theatre

Electronic database

Moral and social responsibility Spiritual moral social and cultural development Values education

Creative thinking Critical thinking Meaning-making Values awareness


Carter and Osler (2000)

Classroommanagement Curriculum Organisation and management Teaching and learning

Citizenship Learners Teaching staff

Evaluation: Researcher-manipulated Controlled trial non-randomised

Secondary school Handsearch Emotional and social literacy Conflict resolution Human rights education

Inter-personal awareness Communication skills Collaboration Problem-solving/decision-making Values awareness


Carter (1999) USA Other topic focus:Resolution of conflict in situations throughout the

The material does not focus on curriculum issues



Other educational setting: Middle school


Conflict resolution Peer mediation

Critical thinking Inter-personal awareness

Personal outcomes Social outcomes

England





school day Intra-personal awareness Problem-solving/decision-making Values awareness

Clare et al. (1996) USA Classroom management Curriculum Organisation and management Teaching and learning

Literature Learners Teaching staff

Exploration of relationships

Primary school Handsearch Moral and social responsibility Spiritual moral social and cultural development Character education

Critical thinking Meaning-making Intra-personal awareness

Cognitive outcomes Moral and political outcomes

Day (2002) England Curriculum Teaching and learning

Other curriculum:Extra curricular


Secondary school Handsearch Moral and social responsibility Spiritual moral social and cultural development Emotional and social literacy

Critical thinking Inter-personal awareness Collaboration Values awareness


Deakin Crick (2002)

England Curriculum Organisation and management Policy Teaching and learning

Citizenship Cross-curricular Geography Hidden Literacy further languages Music PSE Religious Ed.

Learners Senior management Teaching staff Governors


Secondary school Handsearch Moral and social responsibility Community involvement Spiritual moral social and cultural development Character education Values education Service learning

Creative thinking Critical thinking Meaning-making Inter-personal awareness Intra-personal awareness Communication skills Collaboration






Science

Values awareness

Faubert et al. (1996)

USA Classroommanagement Teaching and learning

Citizenship

Learners

Evaluation: Researcher-manipulated: Programme run with four groups: two experimental groups and two control groups for one semester


Character education

Creative thinking Critical thinking Meaning-making Inter-personal awareness Intra-personal awareness

Cognitive outcomes Personal outcomes

Garcia-Obregon et al. (2000)


Citizenship Other curriculum:service learning



Secondary school Unknown Moral and social responsibility Community involvement Spiritual moral social and cultural development Service learning

Experiential learning Meaning-making Inter-personal awareness

Cognitive outcomes Personal outcomes Social outcomes

Hecht and Fusco (1996)


Other: Service learning

Learners


Secondary school: organised by the middle school and taking place in the community, secondary aged students


Service learning Experiential learning Meaning-making Communication skills Collaboration Problem-solving/decision-making Values awareness

Personal outcomes

Horwood and Patterson (1995)

Canada Classroommanagement Organisation and

The material does not focus on curriculum issues.

Learners Description Secondary school Electronicdatabase: ERIC and BEI


Experiential learning

Social outcomes





management Teaching and learning

Johns et al. (2000)

Australia Organisation andmanagement





Inter-personal awareness Intra-personal awareness

Social outcomes

Johnson et al. (1995)

USA Classroommanagement Teaching and learning Other topic focus:Peer mediation



Primary school: Elementary School


Conflict resolution Peer mediation

Inter-personal awareness Intra-personal awareness Communication skills Collaboration Problem-solving/decision-making Values awareness


Johnson et al. (1996)

USA Other topic focus:Mediation programme


Learners


Primary school Elementary school

Electronic database

Moral and social responsibility Emotional and social literacy Conflict resolution

Inter-personal awareness Intra-personal awareness Communication skills Problem-solving/decision-making


Jordan and Le Metais (1997)

Australia Teaching andlearning: Cooperative learning

ICT Learners Evaluation:Naturally occurring

Primary school

Unknown Moral and social responsibility Spiritual moral social and cultural development

Experiential learning Inter-personal awareness Intra-personal






Emotional and social literacy Conflict resolution

awareness Communication skills Collaboration

Kim et al. (1996) USA

Curriculum: Civics education – participatory learning

Citizenship Learners Description Secondary school

Handsearch

Community involvement Political Literacy

Experiential learning Inter-personal awareness Intra-personal awareness

Personal outcomes Moral and political outcomes

Laconte et al. (1993)

USA Teaching and learning





Moral and social responsibility Character education

Intra-personal awareness Communication skills Problem-solving/decision-making


Melchior (1999) USA Curriculum Other topic focusService learning

Citizenship Other curriculum:Service learning


Secondary school Other educational setting: Middle school

Electronic database ERIC and BEI

Community involvement Service learning

Experiential learning Meaning-making Inter-personal awareness Intra-personal awareness


Morais and Rocha (1999)

Portugal Curriculum Teaching and learning

Other curriculum:Pedagogy


Primary school Electronic database: ERIC and BEI

Emotional and social literacy Values education

Inter-personal awareness Intra-personal awareness Communication skills Collaboration Problem-






solving/decision-making Values awareness

Nelson (2003) UK Curriculum:

Citizenship PSE


Secondary school

Handsearch

Moral and social responsibility Spiritual moral social and cultural development

Experiential learning Inter-personal awareness Intra-personal awareness Communication skills


Obenchain (1998) USA Classroom management Teaching and learning



Primary school Electronic database: ERIC and BEI

Moral and social responsibility Community involvement Values education

Meaning-making Inter-personal awareness Intra-personal awareness


Ogden (2000) England Classroom management Teaching and learning



Primary school Handsearch

Emotional and social literacy

Meaning-making Inter-personal awareness Intra-personal awareness Communication skills Collaboration


Pate et al. (2001) USA Curriculum Teaching and learning

Art Science Vocational Other curriculum:Languages

Learners Description Other educational setting: Middle school



Experiential learning Meaning-making


Pipkin and Yates (1992)

USA Organisation andmanagement

The material does not focus on

Learners

Description Other educationalsetting:

Electronic database

Moral and social responsibility

Inter-personal awareness

Personal outcomes





Teaching and learning

curriculum issues.

Teaching staff Middle school Education for diversity Conflict resolution

Intra-personal awareness Collaboration

Social outcomes

Pitiyanuwat and Sujiva (2001)

Thailand Classroommanagement Curriculum Teaching and learning

Citizenship Religious Ed.


Description Secondary school Handsearch

Moral and social responsibility Spiritual moral social and cultural development Character education Values education

Critical thinking Inter-personal awareness Intra-personal awareness Values awareness

Outcomes aimed at rather than reported

Polite and Adams (1997)

USA Classroommanagement Teaching and learning






Moral and social responsibility Emotional and social literacy Values education

Creative thinking Critical thinking Experiential learning Meaning-making Communication skills Problem-solving/decision-making


Russell (2002) Ireland Curriculum Teaching and learning

Other curriculum:Thinking Skills



Handsearch Moral and social responsibility Spiritual moral social and cultural development Emotional and social literacy Values education

Creative thinking Critical thinking Meta-cognition Meaning-making Inter-personal awareness Intra-personal awareness

Cognitive outcomes Social outcomes Moral and political outcomes





Values awareness

Scales (1999) USA Teaching and learning

Citizenship: service learning

Learners Evaluation:Researcher-manipulated


Handsearch Moral and social responsibility Community involvement Spiritual moral social and cultural development Education for diversity Character education Emotional and social literacy Service learning Active Learning

Meta-cognition Experiential learning Meaning-making Inter-personal awareness


Schmuck and Schmuck (1990)

USA Classroommanagement Organisation and management Teaching and learning


Learners Senior management Teaching staff Non-teaching staff Parents


Nursery school Primary school Secondary school

Electronic database ERIC and BEI

Community involvement Political Literacy Values education

Experiential learning Inter-personal awareness Intra-personal awareness Collaboration

Social outcomes

Sewell and St George (1997)

New Zealand Curriculum Teaching and learning

Other curriculum:Social studies


Primary school Handsearch

Moral and social responsibility Community involvement Character education

Creative thinking Critical thinking Problem-solving/decision-making


Stevahn et al. (2002)

USA Curriculum

Other curriculum:Social studies

Learners Evaluation:Researcher-

Secondary school:

Contact Conflict resolution

Creative thinking

Cognitive outcomes





Other topic focus

manipulated American HighSchool

Peer mediation Inter-personal awareness Intra-personal awareness Communication skills Problem-solving/decision-making

Social outcomes

Tibbitts (2001) Romania Curriculum Teaching and learning

Citizenship: Civics


Secondary school Handsearch Political Literacy Values education

Critical thinking Communication skills Collaboration Values awareness


Wade (1994) USA Curriculum Teaching and learning

Citizenship: Human rights


Primary school Electronic database ERIC and BEI

Human rights education

Critical thinking Meaning-making Intra-personal awareness Values awareness



Appendix 4.1: Comparison of studies in the systematic map and in-depth review


Appendix table 4.1: Study types in the systematic map and in-depth review*

Study type Systematic map (N = 35)

In-depth review (N = 13)

Description 5 0 Exploration of relationships 4 3 Evaluation: Naturally occurring 16 5 Evaluation: Researcher-manipulated 10 5 * Codes are mutually exclusive. Appendix table 4.2: Types of citizenship education in the systematic map and the in-depth review*

Type of citizenship education Systematic map (N = 35)


Active learning 1 0 Moral and social responsibility 19 9 Community involvement 12 3 Political literacy 4 1 Spiritual moral social and cultural development 12 7

Character education 8 4 Education for diversity 4 0 Emotional and social literacy 10 3 Values education 12 6 Service learning 6 3 Peer mediation 4 0 Human rights education 4 2 * Codes are not mutually exclusive.



Appendix table 4.3: Types of learning processes in the systematic map and the in-depth review*

Learning Systematic map (N = 35)


Creative thinking 8 5 Critical thinking 12 9 Meta-cognition 2 1 Experiential learning 11 3 Meaning-making 16 10 Inter-personal awareness 25 7 Communication skills 14 4 Collaboration 12 3 Problem-solving/decision-making 11 2 Values awareness 15 7 Intra-personal awareness 23 7 * Codes are not mutually exclusive. Appendix table 4.4: Types of achievement in the systematic map and the in-depth review*

Achievement Systematic map (N = 35)


Cognitive outcomes 17 13 Personal outcomes 27 10 Social outcomes 29 10 Moral and political outcomes 15 9 * Codes are not mutually exclusive.


Appendix 4.2: Details of the studies in the in-depth review

Appendix 4.2: Details of studies in the in-depth review

Beyer and Presseisen (1995) What are the broad aims of the study? Evaluation of a pilot study of Facing History and Ourselves (FHAO), an interdisciplinary moral education programme that uses a case study of the Holocaust on a group of middle school (eighth grade) students in an urban setting. Study evaluated increases in students’ historical knowledge and ability to make connections between historical events and current issues of prejudice, racism and hatred.

What are the study research questions and/or hypotheses? From the programme’s overall goals, this study focuses on two student outcomes: • increase in knowledge of the period of Nazi totalitarianism and the Holocaust, the

causes and the roles played by various people and groups • increase in ability to make connections between historical events and issues and

one’s own experiences and choices Study design summary

This was a pilot study, involving the development of a test instrument to be used before and after a period of collaborative instruction. Two groups were selected: one received the intervention of FHAO interdisciplinary moral education programme over a six-week period; the other group acted as a control group. The same pre-test and post-test was given to both groups and results were analysed with statistical methods.

Type of study

Evaluation: Researcher-manipulated

Findings

• T-tests for specific, general and total mean pre-test scores for experimental and control groups show no significant differences.

• T-tests for independent samples were used to analyse differences between experimental and control groups on three post-tests measures (specific score, general score and total score). Comparisons of all three means were significant.

• After six weeks’ intervention, the experimental group performed significantly better with reference to specific historical content and reasoning about human relationships and responsibilities. However, there was a high degree of variability found for both groups.

• Although there was a large gain in factual knowledge, results show variability among responses: for example, responses to the post-test question ‘What was the Holocaust?’ included ‘a Jewish religion’, ‘ a war against the Jews’ and ‘something that the Jews went through that Hitler started’.

Conclusions After six-weeks of intervention, the experimental group showed a significantly greater increase in knowledge of the period of Nazi totalitarianism and the Holocaust as compared with the control group. Although there was an increase in factual knowledge, the quality of responses indicates areas of concern. Variability of responses suggest some students made little progress; there was little evidence to suggest high levels of student understanding; and many students were unable to use written language to express ideas/thought. Experimental group showed significantly greater increase in ‘general’ terms to do with human relationships and implications of one’s actions (p 4).



Authors find interesting findings by examining content and quality of responses. With reference to issues of friendship and group membership, opinions were tied to their own personal experience rather than based on historical context. ‘Sometimes when you are in a group it’s always someone in that group trying to be bossy’ (p 4). There were some meaningful connections: for example, ‘I learned of the pain and agony of the Holocaust. I learned that slaves were not the only ones who had trials of terror and that African-American and Jews are almost the same and history but most of all I learned that one’s person’s rage can become a world’s rage’ (p 5). Authors also conclude that there were limitations to their pilot study: 1. High morality rate for both groups 2. Relatively short duration (six weeks) of pilot study 3. Importance of writing ability for the short-answer questions They conclude that the pilot was useful to FHAO programme-makers but that goals must be clear to teachers as well as students, issues around student test taking need to be addressed and the survey needs revision with regard to ‘general’ items. The authors also conclude that classroom observation, teacher and student interviews, and a review of implementation data should be introduced.

Black and Goldowsky (1999) What are the broad aims of the study? The study investigates high school students’ reactions to the science theatre play ‘Mapping the soul’ and its use as a means of delivering a part of the science curriculum that has moral and social implications. What are the study research questions and/or hypotheses? The study explores the questions: • Do students differentiate learning science from science theatre as compared with a

traditional science class? • How are specific theatrical techniques, namely conflict between characters and

audience participation, perceived by the audience? • After seeing a production of ‘Mapping the soul’, to what extent are students able to

reason about the social, moral and/or ethical implications related to the Human Genome Project?

Study design summary The study was designed to evaluate the impact and reactions of science theatre on student learning, and ability to reason about social and moral implications of the Human Genome Project. To evaluate the impact a variety of methods were used: • Students were given a pre-performance and post-performance survey. The pre-

performance survey gained biographical information as well as a basic understanding of what students knew about the project prior to seeing the play, while the post-performance survey sought to elicit more detailed information about the students’ reactions to the play by asking open-ended questions.

• There were a number of questions developed after the pilot survey, which meant that two questionnaires were produced and distributed, more or less evenly amongst participants (745 in total).

• While students were completing questionnaires, teachers identified a selection of students that most represented the target participants for interview by the researchers. The post-performance interview lasted 15–20 minutes and was recorded.

• Additional data were collected through performance observations (on audience and play).



Type of study Evaluation: Naturally occurring Findings • Students thought plays were educational, rather than entertaining, and have a role in

educating, informing or teaching science. • Responses to questionnaires indicated that for some science theatre is more

interesting than a science class (28%); gives a sense of real life (15%); is entertaining (21%); is easier to understand (9%); involves the audience (6%); is emotionally engaging (2%); and has a story line (1%).

• The characters were believable and provided credible arguments (76%); 67% of found both characters in the play equally believable.

• There was a statistically significant difference between male and female students who favoured one character above another. The students tended to agree with the character of their same sex. Similarly, the males thought the conflict in the play reflected more positively on the male character, while the females thought the reverse.

• Participants did not join in the discussions during the play. They reported feeling too shy to speak in front of the group (25%); not having enough information about the topic (12%); not wanting to answer (11%); preferring to hear other’s views (8%); or someone else had already answered (4%).

• Although they did not participate, they appreciated the technique of audience participation.

• They were better able to reason about the ways in which the Human Genome Project might influence their lives after the play than before (28% to 87%). After the play, students were able to reason about the impacts of the Human Genome Project.

Conclusions The authors conclude that ‘Mapping the soul’ achieved many of the goals of museum theatre. It presented a complex issue in a way that initiated student’s ability to link together complex scientific and moral issues to personal and societal matters. Students found their experience of learning from a play to be different from learning from a traditional science class. The technique of conflict or argument employed by this Science Theatre production helped to present a multi-sided view of the benefits and concerns raised by the Human Genome Project, although students tended to agree with the character of the same sex in a conflict situation. Although only one-quarter of the students reported taking part in the interactive portions of the play, the majority felt positive about the technique. The students were able to reason successfully about the social, moral and ethical issues presented in this play.

Clare et al. (1996) What are the broad aims of the study? To study instructional conversations in order to understand how to implement extended discourse in the classroom What are the study research questions and/or hypotheses? To study instructional conversations Study design summary The study examined the transcripts of the lessons of a teacher who was systematically assisted to develop a more conversational style of instruction.



Type of study Evaluation: Naturally occurring Findings • Students in the Instructional Conversation (IC) groups that engaged with the moral

issues posed by Mrs Fiske were more than four times as likely to mention more subtle or problematic effects of friendship in essays written after the lesson than were those students who did not.

• In contrast, the majority of students, whose reading comprehension lessons addressed other concerns, showed no evidence that the student treated the events in the story as problematic for a friendship.

• Simply reading the story proved insufficient, indicating that good texts alone may not bring about the kind of cognitive dissonance likely to lead to moral development. Instruction in the form of assisted performance – where a teacher adroitly brings into focus developmentally appropriate questions and helps her charges wrestle with them – may be a key requirement.

• The distribution and emergence of ‘tracer’ points across student texts indicated that it was the students who had been exposed to Stage 3 reasoning, focusing on the importance and meaning of being a ‘good’ friend that enabled them to comprehend the problem in relationships between two characters in the story.

• In aiming to change instruction discourse processes during reading comprehension lessons, the project developed an approach that came to rely on the moral dilemmas presented and explored fiction and other literature to achieve its instructional goals. In exploring these dilemmas, the students in the lessons we examined were able to get more than the basic points from the stories they read. They were able to move beyond superficial understandings to explore the moral and emotional quality of the texts and ‘the embodiment of these qualities in various characters’.

Conclusions The highly interactive and challenging discussion approach of instructional conversation, coupled with Mrs Fiske’s decision to focus on the moral dilemmas of the stories, created a zone of proximal development for the students for both reading comprehension and moral growth. These lessons suggest that moral education can be a useful tool for educators attempting to enhance students’ comprehension. Literature-based infusion of moral education is not easy despite its advantages over an ‘additive’ approach. First, such an infusion requires teachers to be able to recognise appropriate dilemmas and present them to students in a way that is compatible with their moral reasoning. To do so successfully, teachers must be versed in the stages of moral development posited by Kohlberg’s model and must learn to recognise what issues are age-appropriate and likely to create moral dissonance. Teachers also need to develop the technical ability to generate and lead such discussions. This takes teachers away from a conventional role of reciters of wisdom and into a more open, facilitative and conversational pedagogy.

Day (2002) What are the broad aims of the study? A quasi-evaluative study focusing on the experiences and interactions of teachers and students taking part in the Forum Theatre workshop. Forum Theatre gives audience members the opportunity actively to take on the moral dilemmas they are presented with, by entering into the play and taking on the role of one character. They attempt to resolve the conflicts experienced by this character by enacting ideas for appropriate and effective behaviour within the make-believe context of the Forum Theatre stage. What are the study research questions and/or hypotheses? The study was a qualitative case study using ethnographic methods: in particular, constant comparison and other methods to develop codes, categories and themes. Therefore there



was not a specific hypothesis or research question. Study design summary This is a quasi-evaluative study of a ‘unique’ case of drama used for social and moral development. It used qualitative methodology to explore the experiences of teachers and students, and participants in the workshop to identify key themes, codes and categories which reflect the impact of the workshop on students’ moral and social development and on teachers’ practices. Type of study Evaluation: Naturally occurring Findings • All teachers and students found the topic of refugees and homelessness to be highly

relevant because they were familiar from their own experience. • The students attributed the realism of the workshop to the skills and qualities of the

actors, who were of a similar age and background and used language with which they were familiar.

• Different rules applied within the workshop from ‘normal classrooms’: that is, the workshop adopted an interactive approach that invited students to participate as adults, teachers and activists as well as learners.

• Students experienced sympathy towards refugees, which developed into empathy and which they transferred to human experience in general. However, this contrasted with some rhetoric they experienced in their socio-political world in relation to refugees and homelessness.

• Students were able to differentiate between amusing interventions in the story and valuable ones, which were those perceived to make a difference to the outcome.

• The workshop provoked a desire among students to ‘do something about the issues’ in real life.

• The issues raised in the workshop were not followed up in school in a learning context. This led to frustration being expressed by teachers and students.

Conclusions The Forum Theatre workshop is particularly useful for moral education, providing an arena for educating ‘emotional intelligence’. The motivational energy of the empathy stimulated by the play was focused on action: that is, oriented towards the ‘other’. Students would then transfer this empathy to other ‘victims’ they knew in school. The convergence of the experiences of the actors and the students facilitated student identification with the actors. The Forum Theatre provided opportunity for moral reasoning, ‘frame freezing’ and discussions about moral issues. It moved and engaged students, and stimulated them to re-examine their social and moral values. This critical examination process was not subsequently given a forum in school. It enabled a co-operative learning environment, including dialogue and debate. The message of respect for others was conveyed through both the content and the method. Forum Theatre provided a tangible shared experience with moral content on which moral reasoning could subsequently be based. Dramatic methods can enliven personal, social and health education (PSHE), but often PSHE and drama are perceived as separate entities. There was a need for sharing of information about programmes between theatre company and schools in order to facilitate planning. The relationship between drama and moral education is often espoused. The study supports the view (Winston, 1999) that, in reality, drama and PHSE often operate as separate entities.



Deakin Crick (2002) What are the broad aims of the study? This book aims to discover if planned interventions in spiritual, social, moral and cultural education change the way that students perceive, understand and articulate the core set of values underpinning a school’s ethos. The impact of teacher interventions on the students’ general and subject specific knowledge is explored, which develops into a discussion about the impact on teaching and learning styles, and the development of teacher-student relationships. The study attempts to demonstrate that spiritual, social, moral and cultural education does not (and should not) take place as a distinct lesson, but should underpin the whole-school approach to teaching and learning. What are the study research questions and/or hypotheses? • The study has different questions for different stages of the research, but the

overriding questions to be addressed are ‘Can the “softer” side of values and spirituality be managed, measured, improved upon and used as a defensible alternative “voice” in the cacophony of the competing demands and pressures of contemporary school leadership?’ and ‘Are the values promoted by the school, practised?’

• First stage question: ‘Did the school community identify a common set of core values?’• Second stage: ‘Where were values taught in the curriculum?’ • Third stage: ‘What is the impact of teaching core values explicitly within existing

subject areas?’ • Fourth stage: ‘What are the implications for teaching values within the curriculum?’ Study design summary Repertory grid technique, developed by Kelly (1955) within the framework of personal construct theory, was used throughout. Structured interviews gave some indication as to the subject’s core constructs in relation to school. (The construct system is to some degree shared with others and to some degree unique to the individual.) Based on Kelly’s personal construct theory (psychology of persons), it is a grid whose vertical axis comprises elements which represent the area in which construing is to be investigated and whose horizontal axis represents the differing ways in which the subject construes those elements. The elements were representative of the differing aspects of school; these were supplied to subjects for consistency purposes. Elements were identified through interviews with the head and two deputy heads. The reason for this was that, if it was significant to school leaders, it would be significant to the direction of school. The author recognises that this might omit elements of the school that the students and teachers see as the vision. Subjects rated the constructs against the elements. They rated between 1 and 7 with 7 meaning the construct was highly associated with the element. The correlation statistic, Pearson product moment, was used to judge the importance of one construct or element with another. The research team also undertook a curriculum audit in relation to the school’s core values. A member of staff from each department worked with a research teacher to examine the content of the schemes of work in that subject area. It took the form of a discussion about teaching and learning, followed by an interrogation of what and where there were moments of opportunity to focus on the school’s core values. At the beginning of the year, the research team held a working session to identify opportunities to teach values in the curriculum and the teaching strategies to be employed. The teachers then devised a series of lesson plans for two terms. Students completed written work in the main, and teachers completed their observations and evaluations on a fieldwork proforma, which were available for analysis by the whole team. During the teaching, three formal focus groups took place to discuss changes in students’ learning. These were minuted and distributed to teaching staff to comment upon. In addition, the discussions were recorded and analysed by a researcher. Each teacher was observed on two occasions by the deputy head and another researcher; comments



were fed back to the focus group. Two videotapes were also used in the analysis. Each subject area used different types of intervention, which are briefly described in the findings section of each subject area. A control group and an experimental group took part in the same topics in the same subject area, but the control group did not have the values intervention. Students were given a questionnaire (based on the repertory grid technique) to complete at the beginning of the year. The results were entered into a computer program, which produced statistical data identifying the most significant values for each student. Seven months later, the same instrument was used on the same students in the same way, enabling the research team to make judgements about changes in the way students understood school values. Comparisons were made between the control group and the experimental group. After the second tranche of repertory grids was elicited, the data were analysed from a range of perspectives using INGRID and DELTA systems. The mean of each student’s rating score on each construct (1 to 9) was calculated for the first and second grid. The first and second grids were then put through DELTA to provide a correlation statistic measuring the degree of correlation between the grids in relation to the nine constructs. Type of study Exploration of relationships Findings Consultation resulted in nine core values: (1) valuing ourselves, (2) valuing others, (3) justice, (4) forgiveness, (5) truth, (6) trustworthiness, (7) stewardship, (8) fulfilling our potential, (9) faith in Christ. Curriculum audit and pupil perceptions • There were 286 constructs in total, of which 35 (or 12.2%) related to positive

interpersonal relationships. Values relating to friendship were more important than others, signifying an alternative conception of the ‘autonomous self’ from the conception of ‘persons in relation’ (Gilligan, 1982).

• Twenty-nine constructs related to self-identity (10.1%), which overlapped with equality which numbered 17 (5.9%). However, the most significant family of constructs were those related to learning, achievement and excellence (65, 22.7%).

• Thirty-four (11.9%) related to Christian faith and 25 (8.8%) related to moral development or knowing right from wrong.

• The environment was important to students with 26 (9.1%) responses received, but constructs related to the curriculum were scarce (1.1% or 3).

Teachers and staff • There were 62 constructs in total. As with the students, learning and achievement was

the most significant family of constructs (14 or 22.6%). Valuing interpersonal relations and teamwork were identified in 9 (14.5%) of the constructs. This is consistent with the findings from the students on the value of interpersonal relationships.

• Valuing equality in terms of meeting the needs of all students was found in 7 (11.3%) constructs, with 11 (17.7%) relating to the environment, in particular the learning environment, in terms of resources and space.

• Eight constructs referred in some way to the Christian foundation of the school, most commonly in terms of its historical roots and its influence on the ethos and mood of the school, but not directly related (or relevant) to the content.

• Five (8.1%) constructs related to the curriculum in terms of the breadth and scope of it rather than the content.

Overall, this stage of the research confirmed and supported the findings of the values consultation, highlighting the importance of positive relationships for both teachers and students and a desire for all students to achieve highly. Interestingly, the curriculum was highlighted as needing further development and commitment in terms of delivering spiritual, moral, social and cultural education.



After the planned intervention, the results were as follows (by subject area): Science: Students conducted an experiment of working in groups over the course of three one-hour sessions. At the beginning of the last lesson, the teacher introduced the nine school values for students to consider in relation to the experiment. The results were recorded through a brainstorming session and include the following: respecting the comments of others, listening to others ideas, sharing equipment, not making up results, forgiving other people’s mistakes and trusting people. The next intervention in science was focused on blood transfusions. This took place over a week and included homework. Two worksheets were completed; the second was given as homework a week after the first and revealed how Charles Drew died from a road traffic accident and was refused a blood transfusion from a whites-only hospital. The students’ responses were quite emotive. Correlations were found between the way students dealt with the values prior to the intervention and after, resulting in the view that values intervention encouraged a reflective searching for deeper meanings to events and issues. They also encouraged student responsiveness. French: 14/15 year-olds, of average ability who were a little disaffected with French (especially the boys). Most of the sessions were focused on real-life examples, thus appearing to deal with values, but students did not initially appear to make the links between what they were doing in class and a real-life scenario that would relate to them. The students were given a test and an IT project which provided a summary of what had been learned. In a topic that required students to decide on what they wanted in their community, respect, justice and valuing others emerged as important considerations. The rules that the community would abide by enabled the school’s core values to be considered in greater depth and application. In a topic about food and drink, questioning that elicited deeper meanings was characteristic of the intervention and resulted in discussions about underlying values and prejudices about foreign food and cultures. When discussing the values, the teacher related them to life in the school, providing a ‘lived’ perspective. However, the development of language skills was considered to be restrictive, leading to considerations of developing the language skills from Year 7. Religious Education: 14/15 year-olds, middle ability. Social class, gender, ethnic origins and free school meals register were used to ensure that the group reflected as far as possible the most holistic view of students attending the school. Having recently explored the notion of commitment, students were required to think about their personal journal and consider what type of commitment was required at each stage. Explicit intervention about the how the Christian faith was a prerequisite was then considered. Students voluntarily offered baptism and marriage in church as important stages in their personal journey. A story about baptism followed and students discussed the importance of being able to practise a religion so long as it was ‘acceptable to a tolerant society’, which lead on to a consideration of justice. Geography: Age 11/12 years, mixed ability, but in reality above average. Values targeted were stewardship and respecting others. This took the form of discussion, through role-play, about a new road being built. Lessons were recorded on video and students were reminded of respect for others when becoming passionate about their cause! In another intervention, students examined the shape of the earth, where they were given a story about the western understanding of the earth’s shape. This led to a discussion about Galileo’s punishment for his mathematical theories, while also accepting the church and God. Students wrote their reflections in silence and then read them aloud to the class. On a field trip, students identified trust as a key value: trusting the boat driver and teachers, and teachers trusting the students. The theme of forgiveness arose in a session that required the interpretation of photographs, written and oral anecdotes about the rebuilding of Bristol after bomb damage. There were comments that noted the need to forgive some actions of others in



order to move forward in life. The explanation of values was found to have added an extra dimension to the curriculum, involving a spiritual, reflective approach to discussions which built up trust and respect through the sharing of ideas. Music: 13/14 years, mixed ability. Having composed a piece of music the students were involved in a peer-assessment exercise which involved discussion of truth, sympathy and empathy, and involved skills in listening and respect for others. The lesson was conducted through process not content, as the content is individual. In a group work activity in music, the teacher initiated a discussion of values related to teamwork and the class engaged in a brainstorming session about which skills and values were required to work in a team. At the same time as discussing the value of teamwork, the students had to display the appropriate skills in order to meet the outcomes of the lesson. The students found that it was important to treat people equally and fairly, to listen to others and to appraise them. This had an impact on the way they worked in groups in other lessons as this session was used as a point of reference, thus resulting in the values being used as a tool to transfer learning. Conclusions The quality of thinking and talking was significant to the whole process in enhancing the learning that took place. The intervention addressed the whole person as a learner and resulted in teachers relating to the students differently. Consequently, the teachers felt that a ‘one off’ PSHE slot was inappropriate, preferring an environment in which the teacher meets the students in other contexts. Thinking skills went beyond what would normally occur in the subject lessons. Teachers discussed the two years’ research and came up with three groups of outcomes: • Student development: Values intervention provided a vehicle that enabled students to

make sense and meaning out of their learning; a gap was identified between theory and practice, which requires further spiritual, moral, social and cultural development.

• Making connections: Dialogue is the most appropriate medium to explore this gap, so that students can name and speak of their experiences. The researchers state that the importance of quality discourse cannot be underestimated. Discussion of values resulted in some instances in an energy to embark on action related to the chosen topic. This fits well with the citizenship aim to enhance community involvement.

• Holistic curriculum: Values intervention adopts a holistic relationship between students and teachers, rather than concentrating on just the cognitive needs. Teachers involved felt that, in order to deliver the interventions well, they needed to move beyond their normal teaching resources and expand to consider global issues, thus resulting in professional development. This led them to consider their teaching to be more purposeful as it provided an added dimension to their subject.

Overall, the research demonstrates the ability to operationalise an approach to spirituality, that is integrated with everyday life and experience.

Faubert et al. (1996) What are the broad aims of the study? The study aimed to modify a classroom environment through a programme called Getting on the Right Track and to explore whether that resulted in promoting abstract thinking and ego stage development among students. What are the study research questions and/or hypotheses? What is the effect of a role-taking action learning programme on the cognitive and ego development of African-American rural high school students? Study design summary The study is a quasi-experimental, researcher-manipulated design, with an experimental group which received the intervention and a comparison group which did not. Pre- and



post-tests were used to measure the impact of the intervention. Type of study Evaluation: Researcher-manipulated Findings In the analysis of the cognitive data, the covariate was the pre-test Sentence Completion Test (SCT) score. The main effects were gender, school, grade and group. • The only statistically significant main effect was for group (F(I,I)=4.17, p:50.05),

although there was a tendency toward statistical significance for gender (F(I,I)=3.36, p:50.07). As expected, the experimental groups receiving Deliberate Psychological Education (DPE) were found to demonstrate significantly greater gains in cognitive abilities (controlling for initial cognitive skills). The main effect for males in the experimental groups was the greatest. Neither gender, school nor grade yielded statistically significant main effects, but none were expected.

• There were statistically significant interactions between gender and group (F (I,I)=5.39, p:50.02). There was a trend toward a three-way interaction among gender, school and group (F(I,I)=345, p:50.07). No other interactions were found between or among any other of the 11 possible combinations of interactions and none were expected.

• In the analysis of the ego stage data, pre-test scores were used as the covariate. The main effects were gender, school, grade and group. The only statistically significant main effect was for group (F(I,I)=17.81, p:50.05), although there was a tendency towards statistical significance for gender (F(I,I)=3.20, p:50.08).

• As expected, the experimental groups receiving DPE were found to demonstrate significantly greater gains in ego development (controlling for initial ego stage).

• The main effect for females in the experimental groups were the greatest. Gender, school or grade did not yield statistically significant main effects, and none were expected.

• There were no statistically significant interactions, although there was a trend toward a three-way interaction among gender, school and grade (F(I,I)=3.50, p:50.06). No other interactions were found between or among any other of the 11 possible combinations of interactions, and none were expected.

• ‘The results indicated statistically significant positive changes in both formal operations and in the level of ego-development’ (p 541).

The two statistically significant findings (and the two tending towards significance) supported positive outcomes from the programme. The authors also present findings from what look like qualitative analyses of students’ responses to the programme in the results section. Reference is made to an unpublished doctoral dissertation by the lead author. This would need to be seen to support use of these findings as information for this systematic review. Conclusions The two major empirical measures supported positive outcomes from the Getting on the Right Track project. On the assessment of cognitive growth from concrete, literal thinking towards the ability to think abstractly and scientifically, the results indicate significant gains. This shift from concrete to abstract thinking usually takes a long time (4/5 years; Renner et al., 1976), thus these findings support the effectiveness of the DPE. The results suggest that it is critical for African-American rural youth to have an opportunity to develop higher order intellectual skills. The current programme offers evidence that this DPE intervention can be considered a step in the desired direction for the development of cognitive growth. ‘On the second measure, an instrument of self-concept/ego development, the results have similar indications. At higher levels of ego development, adolescents begin to demonstrate greater autonomy, more reliance on self, a firmer sense of identity and greater reliance on independent problem-solving (Loevinger, 1976). It is an arguable point as to which is more important, building academic skills or promoting self-reliance. The authors view the



solution as interactive. By helping students develop their thinking skills and simultaneously helping connect these to their valued heritage, we do not have to choose which is more important; both are equally relevant. Self-reliance and a strong ego by themselves cannot make up for cognitive deficit, and cognitive growth cannot generalize to self-reliance and a strong ego. In fact, we would urge that future innovative educational programmes seek such dual goals as major objectives’ (p 541). It was clear, based on qualitative and subjective observation, that the students in the experimental classes became more potent learners and more self-confident as students. For example, one student wrote that Getting on the Right Track helped them to ‘know what my goals would be for the future’; another declared that Getting on the Right Track helped them to ‘develop my skills in which I can be prepared for college’; another stated that Getting on the Right Track helped them to ‘think more on my values in life’; and, finally, one participant wrote that Getting on the Right Track helped to broaden their horizons. In addition, the qualitative findings of greater active participation from class officers and school club members, and other heightened involvement in the school, supported the quantitative findings. The African-American rural youths, who participated in the planned programme, manifested their motivation by enthusiastically attending the planning and reflection sessions, taking their responsibilities toward one another seriously, working diligently on their African-centred research projects, and presenting them with poise and competence at the science festival. The positive attitude and behaviour of the students provided evidence for these African-American rural adolescents’ growth in the cognitive and ego domains. The African-American rural youths, who participated in this DPE, independently sought to know as much as they could about Africa and African-Americans. They asked penetrating questions about why some African-American students did not do well on standardised tests and why such tests often did not include them among the norm groups. They explored the reasons why it was appropriate to name themselves ‘African-American’. They investigated the contributions of Africans and African-Americans to mathematics and the sciences. They investigated mathematical and scientific questions. The current study was designed as an investigation of a role-taking, action-learning programme for African-American secondary students in a rural setting. The programme focused on thinking skills in science connected to African-American and African contributors. The goal was two-fold: to promote an increase in cognitive formal operations and to increase personal ego strength and autonomy. The results indicate statistically significant positive changes in both formal operations and in the level of ego development. Extensive subjective and qualitative narratives cross-validated the empirical outcomes.

Garcia-Obregon et al. (2000) What are the broad aims of the study? The purpose of this study is to evaluate Community Connection, a school-based service-learning programme implemented in a local South Texas middle school. The researchers wished to determine whether the programme was accomplishing its stated objectives and impacting student achievement, student attendance, student perception toward self, school and community. What are the study research questions and/or hypotheses? There were five specific research questions: • Do students participating in a school-based service-learning programme show

achievement gains in reading and mathematics over the previous year as measured by Texas Learning Index (TLI) scores on the Texas Assessment of Academic Skills test (TAAS)?

• Do students participating in a school-based service-learning programme have higher attendance rates than in the previous year, as measured by official attendance records?



• Do students participating in a school-based service-learning programme have lower incidences of behaviour problems then the previous year, as measured by discipline referral records?

• Do students participating in a school-based service-learning programme have a positive perception of self, school and community, as measured by a student survey?

• Do teachers participating in the Community Connection programme have a positive perception of the programme, as measured by teacher survey?

Study design summary A multi-method approach was used in this study to collect both quantitative and qualitative data. The combination of methodologies consisted of a student survey, teacher survey and field research. Two surveys using Likert scales were used. Data were collected from 120 students in one middle school and 19 teacher surveys in order to: • quantify the responses of Community Connection teacher and student perceptions of

the programme’s effectiveness • provide information about the school’s programme in regard to achievement,

attendance, behaviour, attitude, and parental involvement The qualitative data gathered through the field research was used to gain insights into the meaning of the quantitative survey results (p 24). Alongside survey data, the researchers collected additional information from attendance records, disciplinary referrals and TAAS. An unspecified number of students were given before and after measures. Type of study Evaluation: Naturally occurring Findings Results for students who participated in the Community Connection programme: • Students made increased gains in academic achievement, as shown by a significant

gain in the Texas Learning Index scores and a decrease in failure rate on the TAAS test for reading and mathematics. There was an increased pass rate of 17.1% to 92.9% pass for the intervention cohort (p 27).

• The number of absences decreased for the intervention cohort compared with the same students the year before. The percentage of students with eight or more absences decreased from 39.8% to 36.3% in the year of the programme (p 28).

• Students had a lower number of referrals for misbehaviour during the year of the Community Connection programme. The number of students with four to seven referrals dropped from 17.7% to 8% and those with more than eight referrals dropped from 21.2% in 7th grade to 6.2% in 8th grade when they took part in the programme (p 29).

• Similarly, the number of students with none to three referrals increased from 61.1% to 85.8%, suggesting that students were managing to contain their behaviour and were receiving far fewer referrals in the year of the programme.

• An analysis of the qualitative data revealed an impact on the students’ affective domain. The students reported feeling better about attending school and better about themselves. They felt that the programme had helped them improve their behaviour, helped them get along with others and helped them care about helping others. They ‘felt good’ about participating (pp 29–31).

• Teachers had strong support for, and a positive perception of, the effectiveness of the Community Connection programme. They reported that the programme had a positive impact on students’ lives, especially their attitude towards themselves.

Conclusions The authors conclude that the evidence supports a view that the Community Connection programme was effective in Baker Middle School in the four areas identified for students and in the perception of the teachers. The quantitative and qualitative data support a conclusion that the programme was impacting students’ academic achievement, their



attendance, their behaviour and their perception of themselves, school and community. Ninety per cent of the students reported that the programme had helped them ‘care about others in need’. The data also suggested that teachers viewed the programme positively.

Laconte et al. (1993) What are the broad aims of the study? This study set out to investigate the impact that a developmentally appropriate affective education programme had on the self-concept and grades of middle school students (p 1). What are the study research questions and/or hypotheses? The implicit research question is: what is the impact of the programme on the self-concept and grades of rural middle school students who were classified as seriously emotionally disturbed and being high risk for dropping out? Study design summary The study sought to investigate the effects of an affective curriculum specifically ‘Thinking, Feeling, Behaving’ on the self-concept and grade achievement of a group of emotionally disturbed adolescent students identified to be at risk of dropping out of school prior to the completion of High School. The study took the form of an experiment with an experimental and control group. Of the 23 identified students, 13 were randomly allocated to the experimental group and 10 to the control group. A post-test only measure (Tennessee Self-Concept Scale) was administered to the experimental and control groups at the end of a 16-week period during which the experimental group had been part of 15 sessions of the programme. Type of study Evaluation: Researcher-manipulated Findings • There was no difference between the experimental and the control groups in self-

concept scores. • There was no significant difference in grade scores for the experimental and control

groups (p 278). • Specific details were taken from the experimental and control groups in the two

schools and by gender (in one school). Conclusions No significant differences were found between control and experimental groups on total self-concept, components of self-concept and grades. The authors discuss in some detail some possible reasons why no significant differences were found on the test and grade reports after the programme intervention. They suggest six factors: lack of time in teaching sessions; ‘outside circumstances’; characteristics of the student group such as low self-esteem (not measured by pre-test); small sample size; lack of effective implementation of the programme; and lack of effect of this programme as compared with other possible programmes. These results for middle school students are consistent with results found by Strein (1988) using elementary students. Strein found that there is a lack of positive significant findings in the more carefully designed studies, especially for programmes with an internal focus, such as self-concept.



Melchior (1999) What are the broad aims of the study? This is an evaluation of the Learn and Serve programme which has the following goal: ‘The primary goal of the Learn and Serve program is to help young people develop as responsible citizens, improve their academic skills, and develop as individuals through involvement in meaningful service linked to structured learning activities’ (p 7). What are the study research questions and/or hypotheses? What is the impact of programme participation on programme participants? How have Learn and Serve programmes affected the civic, educational, and social skills and attitudes of participating students? This question was broken down into three parts: • What was the impact of service learning on participants’ civic development? • Did service learning help to build students’ understanding of their communities, their

sense of social responsibility and their commitment to community involvement? • What was the impact on educational development and academic performance? Did

service learning increase students’ engagement in school, school attendance and/or academic performance?

• What was the impact of service learning on students’ life skills (such as communications skills, work orientation, and career awareness) and did it lead to a reduction in involvement in risk behaviours?

Study design summary ‘The impacts from the programs were estimated by comparing the average outcomes for program participants with those of comparison group members after making adjustments through regression formula for differences in both baseline scores and the baseline characteristics of the two groups’ (p 7). The summary of post programme participants used data from survey and school record data for a sample of 608 programme participants and 444 comparison group members. Follow-up data included surveys from 764 participants and comparison group members (460 participants and 304 comparison): 66% from high school and 34% from middle school. School record data were also collected from 596 students. The evaluation focused on 17 school programmes, ‘across the country’ [implicit USA], using a variety of quantitative and qualitative methods to analyse survey data and school record information for approximately 1,000 programme participants and comparison groups, surveys of teachers at the 17 schools, telephone interviews with staff at community agencies, and onsite interviews and observation of programme activities. This was in order to measure the impact of the programme on participants’ civic development, educational development and academic performance, impact of service learning on students’ personal and social development. Participants were examined three times: once before programme and at two impact points after – one at the end of the programme to measure short-term impact and the other one year after the programme to measure long-term impact. For the latter, assessment was based on a combination of participant surveys and data drawn from school records. The evaluation team also collected information on participants’ responses to their service experiences through surveys and through interviews conducted at end of programme year (1995–1996). Type of study Evaluation: Naturally occurring It is an evaluation of policy implementation and the effects it has on student learning, but the researchers did not implement the intervention.



Findings Two sets of results relating to the two points in time of data collection are presented. The post-programme results are presented below. Civic/Social attitudes • Statistically significant impacts were reported on three out of four categories in this

area: cultural diversity, service leadership and overall measure of civic attitudes. These results relate mostly to the high school students. There was a less than 5% difference between participant and comparison groups regarding civic attitudes and the author comments that most of the young people started with well developed civic attitudes, so perhaps service learning can be seen as a strengthening process.

• Large impacts were reported on students’ attitudes towards service itself, with a particular focus on whether they felt they could make a difference.

• Significant impact was reported on volunteer behaviour: participants were 20% more likely to be involved in volunteer behaviour than comparison groups (overall). High school students were three times more likely to volunteer.

Impacts on educational attitudes and performance • Overall, there were positive statistical impacts on two measures – school engagement

and mathematics grades – with slightly broader results for the high school students. Other statistical results show that gains were incremental, but not significant. However, the author comments that service learning generally has a positive impact in school engagement.

Social and personal development • Overall, there were no significant post-programme impacts, but there was some

evidence that middle school students were less likely to be involved in risk behaviours. Impacts among sub-groups • No one group benefited more than others overall, but there was some evidence to

suggest that non-white students showed stronger impacts on measures of academic achievement.

Participant perspectives • Statistical evidence was found to show that this was a positive experience that

impacted on their future career choices and understanding of communities; the students felt that they learned more than they would in a class.

Participant impacts one year on • Most had disappeared. There were marginally significant positive impacts on school

engagement, service leadership and science grades. High school and middle school differences • High school students showed positive, statistically significant impacts on service

leadership and science grades, and marginally significant impacts on school engagement and hours of volunteer service.

• Middle schools students still showed an impact on the level of arrests. Conclusions The author reports significant benefits for schools, communities and students in service learning programmes, but it is not the vehicle for large-scale educational change. However, some of the strong impacts on making students more aware of their community and engaging in school more fully, and the reduced likelihood of middle school students to engage in risk behaviours, does provide evidence that students benefit from service learning opportunities within the curriculum. There is a strong case for service learning as a tool for civic and educational development of middle- and high-school students. The cost was low; it helped strengthen civic attitudes, volunteer behaviour and school performance; and provided needed services to community (p 26). Results from ‘well designed’ programmes suggest programme quality makes a difference



– well designed meant the higher likelihood of a significant impact – with the Corporation for National Service to continue emphasis on and disseminate best practice. Highlights a need to define a clearer set of goals and expectations for integration of service into schools and curriculum. Suggests need for continued research on longer term and cumulative impacts of service learning. Recognises study is only one step toward improving understanding of impacts and effective practices, and makes a strong case for the effectiveness of well-designed service learning programmes.

Polite and Adams (1997) What are the broad aims of the study? The study’s aims were to determine whether the Socratic seminar methodology has significantly affected students’ and teachers’ attitudes toward learning at Lookout Valley Middle School. What are the study research questions and/or hypotheses? The impact of a Socratic seminar approach on students’ learning Study design summary The study was a qualitative inquiry using nine teacher interviews and a random sample of 34 students, using a semi-structured interview format. Type of study Evaluation: Naturally occurring Findings The results fall into several categories: (1) teachers’ perspectives, (2) student perspectives, (3) general findings on the seminar environment. Teachers’ perspectives • Seminars were planned on at least three levels: district, team and individual. Team

planning was not taken seriously by all the teams and teachers (p 262). • It was not clear that higher order thinking was always a primary and governing part of

the teacher planning (p 262). There was a need for an operational definition of the Socratic seminar method to explain the rationale that supports it. There was confusion regarding the advantages of seminars over traditional class discussions (p 265).

Student perspectives • Conflict resolution: The use of Socratic seminars was strongly associated with

students’ conflict resolution skills. The level of conflict resolution expertise demonstrated by the students in the study was significantly higher than would normally be expected from a middle-school population (p 266).

• Results showed stratification by gender: 66% of girls used conflict-resolution schemes and 50% of boys employed advanced conflict-resolution strategies. One cannot discount that girls were more mature than boys.

• Higher order thinking skills: Approximately 80% of the student sample engaged in at least intermittent meta-cognitive activity or Piagetian formal operational activity. This was unusual, as sixth-graders would usually be functioning at the level of Piagetian concrete operations. This finding applied to both boys and girls, although the girls scored slightly higher than the boys.

• Meta-cognition: Approximately 40% of the students attempted to use meta-cognitive skills throughout their life experiences; 60% perceived meta-cognitive skills as valuable outside the seminar environment. There was a maturation differential between the boys and girls. No girl reported having no understanding of the uses of meta-cognitive skills.

• Conflict resolution recall: Very few were unable to recall any of rules; students



perceived these to have universal value. General findings • Best seminar experience: Relevance to own life experience; interesting; majority of

class actively participated. • Worst seminar experiences: Topics that were highly abstract – incomprehensible or

weird; irrelevant topics; general disinterest – 25%; significantly more boys said that metaphorical content didn’t make any sense.

• General perceptions: 50% strongly preferred seminar environment to regular classroom because they felt they accomplished more; felt thoughts and feelings were taken seriously in seminar; and seminar topics were always more interesting. Fellow students seemed more polite in seminar. A minority disliked the setting and expressed a preference for structure and authority figures, saying they could only learn when teacher was dispensing information. Only 5% of the students indicated that their favourite seminar topics were largely metaphorical in nature. All the positive responses to metaphorical topics came from boys and none from girls.

Conclusions The authors conclude that there is evidence to suggest that Socratic seminars have the potential to increase the cognitive and social functioning of middle school students. Teacher interviews revealed significant dissonance among instructional staff with regard to how the seminars differed from regular class instruction. The student interviews indicated that Socratic seminars provide students with ample opportunities to develop a number of cognitive and social skills commonly regarded as key goals for children at the middle /school level (p 275). There was significant evidence that seminar discussions were effective in engaging students in tasks that called on their meta-cognitive and critical thinking abilities, while simultaneously developing both conflict-resolution skills and an increased respect for the opinions and feelings of their peers. The authors concluded the article with a list of recommendations for local educational decision-makers. (1) Involve key school administrator for planning and implementation. (2) (a) Encourage team members to plan seminars based on students’ interests and (b) compel students to use higher order and abstract thinking skills. (3) Encourage cross-grade planning (holistic approach). (4) Provide teacher workshops that (a) define meta-cognition, (b) emphasise importance of meta-cognition to seminars and (c) establish means for linkage to occur. (5) Provide training opportunities. (6) Stress importance of understanding distinction between traditional classroom teaching that incorporates focus group and discussions sessions from seminar approach. (7) Encourage each interdisciplinary team to assess its development as a team. (8) Ensure all teachers value and understand the exploratory needs of students. (9) Appraise parents of general concepts and theories behind the seminars from outset of implementation.

Russell (2002) What are the broad aims of the study? To illuminate and interpret children’s moral consciousness and the part played by discussion in helping to foster moral development What are the study research questions and/or hypotheses? The study sought to find categories of meaning and propositions that sum up main themes, which emerge from transcripts of children’s conversations with each other and



their deliberations and reflections on moral and social dilemmas. Can a ‘community of inquiry’ enhance moral awareness in young people? Study design summary The study used the constant comparative method to sort through transcripts of children’s moral deliberations within Thinking Time sessions. There were two groups of children: one aged 7–8, and the other aged 11–12 from two schools. These were analysed for key themes, meanings and patterns. Type of study Evaluation: Naturally occurring Findings • Fairness was a theme that was central in the children’s discussions. Children tend to

begin with broad definitions and then refine their ideas following interaction with others. ‘Fair’ was first defined as quantitative equality, then expanded by the older group to include forgiveness and reciprocity. The older children used more deductive reasoning and made more distinctions. Both groups used utilitarian considerations.

• Boys in the younger group were adamant that rules must be kept for fairness, then later saw fairness as equity taking account of and accommodating individual difference. The girls saw fairness in terms of sharing an awareness of another’s need.

• Responsibility and choice was another theme. One must do what is right and take responsibility for that decision. Choices demand courage and a sense of responsibility.

• The value of human life takes precedence over legal considerations when the two are in conflict.

• Empathic and impartial reasoning: both boys and girls used impartial and empathic reasoning interchangeably depending on the context.

• Gender differences: Younger boys were more concerned with keeping rules within fairness, whereas the younger girls were more concerned with equity. Sharing was an important part of girls’ play; they made no allusion to rules in playing again. Girls felt an obligation to report damage accidentally caused, boys did not.

• Authenticity: There was a strong leaning in the data for all children to be included to choose the good for its own sake. Authenticity means one had to do what one thought was right, even if that conflicted with authority.

• Respecting others: There was a consensus in both groups that one must respect others and that involved the inclusion of the marginalized in society, according them equal rights on the basis of our common humanity. Exclusion was unfair and unacceptable. Colour or race did not have special connotations for the younger group.

• Both groups probed the meaning and attributes of true friendship. One must possess the qualities that one demands in a friend and one only knows oneself through relationship with others. Loyalty and trust are hallmarks of true friendship for the older group. There was a high level of moral awareness and developing sensitivity in the groups.

• Older children’s superior ability to articulate enables higher levels of reflections. There was a high level of participation by children who have learning difficulties in other areas. Having a voice, they felt that what they said mattered and others listened.

• There were different levels of thinking evident between older and younger groups. The more able children provided ‘scaffolding’ for the less able and challenged their thinking. Children at the highest level of ability raised the level of the discussion.

• Children displayed different levels of thinking at different stages of the discussion. Towards the end of the discussion, children would make statements, which attempted to clinch the meaning.

Conclusions The conclusions of the study emerge from the headings of the data transcripts which are



presented in the findings. The overall conclusion is that dialogue has a role to play in fostering reflective thinking, understanding, tolerance and respect for others. It demonstrates the importance of student voice. This study has implications for pedagogy, which the author lists as change in the role of teacher from didactic methods to dialogue; improved relationships; fostering inclusiveness; transfer of skills to other areas.

Tibbitts (2001) What are the broad aims of the study? Aims of the study are not clear. The reviewer infers from results discussion (p 38) that the purpose is to evaluate instructional methodology and the experimental civic education programme (p 29) on the development of critical thinking, dialogue and participatory methods of instruction (p 27) and on participatory attitudes or ‘civic behaviour’ in students. What are the study research questions and/or hypotheses? There are no specific questions, but it is implicit that the study is comparing students who used the experimental texts with those in same school who received civic instruction using the official Ministry textbook. Study design summary Quasi-experimental design: four classes, comprising 109 students in Grades 7 and 8, in three locations in Romania, receiving the experimental civics project. Students were administered a questionnaire. In addition to these four classes, comparison classroom groups in the same schools, but not receiving the experimental civics texts, were administered the questionnaire. Questionnaires were administered four times over two years: in Autumn 1994, Spring 1995, Autumn 1995 and Spring 1996. A total of 222 students in both treatment and comparison classes were administered questionnaires. The paper summarises the results of the questionnaires. Type of study Evaluation: Naturally occurring Findings Qualities of good citizen • Significant gains were seen in the treatment group over the comparison group in two

qualities: voting in most elections and trying to influence government decisions and policies.

• Both groups’ ratings of the importance of several values fell but not significantly: honour one’s country, work hard, be a good human being, be helpful and friendly to others, and do not bring dishonour to the country.

• In all but one item, decreases were higher in the control group than treatment group. • Both groups showed increases (not significant) in obeying the law, volunteering for

service and keeping informed. • Regarding differences between the sexes, the only statistical difference was an

increase in males on the importance of voting in elections compared with females - (F=4.56, p<0.034).

• Qualitative coding showed that groups had parallel increases in the valuing of ‘obey the law’ item over two-year period.

• Each group showed a 37% increase between initial and final survey. • Both groups showed a decrease in value of overall civility. • Consistent with statistical analyses, treatment group showed increases in voting and



participation in public life. For the control group, there were decreases in being honest and working hard.

Importance of human rights • Control group showed significant increases in ratings in civil and political category:

equality before the law, freedom of opinion and expression, freedom from torture and cruel and degrading punishment.

• Treatment group showed significantly larger increases in ratings of two human rights: the right to a job and the right to leave the country and return.

• Two significant differences between male and female: female increase in right to have a job (F=3.93, p<0.049) and right to have basic needs met (F=4.52, p<0.035); there were decreases for males in these categories.

• Correlations show increases in the importance of nine human rights for treatment group only.

Conclusions Study confirms clear link of instructional methodology and the development of participatory attitudes, or civic behaviour in students. Result provides room for optimism, given political context and traditional teaching methods in which the experimental lessons occurred. This was a small study and raises questions. It would be interesting and important to see if participatory attitudes for the treatment group are sustained. The study shows that students in this age group generally become more aware of the role and rule of law, expanding notions of citizenship to beyond that of civility and good manners. This was reflected in increases for both groups in valuing respect for the law. This may well be a reflection of political socialisation of early adolescents in Romania. The results also demonstrate that students hold multiple ideas, values and predispositions in the area of citizenship education. It suggests that a textbook methodology can influence a particular domain of citizenship education; it also shows that a textbook, using more conventional teaching methods, can influence the valuing dimension. A disappointing aspect was the finding in the human rights area: there is no clear pattern in terms of higher valuing of human rights overall. Both experimental and Ministry texts devote only a modest amount of attention to human rights issues, although methodology in the experimental group was seen as supportive of skills conducive to respect and promotion of human rights values. An implicit result is that two years and considerable teacher support was necessary for this result; it is possible where technical support is provided and teachers are open to change. Even under such circumstances, changes in student attitudes do not happen quickly; statistical differences did not emerge until two years into the programme. Students’ increased valuing of participatory dimensions of citizenship does not reduce their loyalty and sense of affiliation with the state. The study would need to be replicated on larger scale and perhaps in different democracies to see what findings prevail.

Wade (1994) What are the broad aims of the study? The study focused on understanding the conceptual change processes of 17 fourth-grade students as they attempted to make sense of the abstract concept of human rights. It sought to develop a deeper understanding of why students did or did not develop accurate ideas about human rights following a month-long unit of instruction. The focus of the analysis was to determine the cognitive, motivational and contextual factors



influencing the changes in students’ understandings of human rights. What are the study research questions and/or hypotheses? A re-examination of qualitative case study data to develop a deeper understanding of the evolution of students’ conceptions of human rights Study design summary Action research One fourth-grade classroom in a rural public school in New Hampshire from October 1990 to June 2001; nine girls and eight boys, ranging in age from 9 to 11 Teacher-Researcher collaboration (1) Researcher entered the class in October 1990 and assisted the class teacher as participant observer until January 1991. (2) January 1991: researcher took a more active role by working with the teacher and students to develop democratic classroom practices. Researcher taught for a month-long unit on human rights and worked with students on social action projects throughout the remainder of school year. Data were collected during this time. Type of study Evaluation: Naturally occurring Findings A number of findings support prior research indicating that upper elementary students are concerned about rights and fairness in their daily lives, yet have little understanding of the concept of human rights initially. • Many of the students brought misconceptions about human rights to the classroom, in

particular interpreting human rights as having the freedom to do what you want. The word ‘freedom’ had been prominently featured during their study on civil rights and could have played a part in the students’ misconceptions.

• Some students were motivated by their own life experiences to learn about pertinent human rights issues.

• Students who had some accurate schemata to support further knowledge development about human rights and task-mastery goal orientation to learn were the most successful.

• In the face of conflicting information, students relied on strongly held patterns of thinking, based on their prior knowledge and experiences. It is not clear whether these students ignored the information presented or simply did not recognise any conflict between their initial ideas and the unit activities.

• The findings point to the role of emotional salience in fostering motivation to learn and subsequent cognitive engagement as an important area for further research.

• The structure of learning activities affected motivation and cognitive engagement among students and, in general, students were highly motivated during the unit activities.

• Placing low emphasis on grades and evaluation provided students with many opportunities for choice and control in their learning, and presented a variety of tasks to challenge and interest the students (Ames, 1992). Cooperative learning, teacher enthusiasm, social interaction, games and simulations, finished products and connecting activities to students’ lives were all central features of the curriculum (Ames and Ames, 1984; Brophy, 1987; Johnson and Johnson, 1985).

• To promote cognitive engagement, teachers must challenge students’ misconceptions and teach them in order to illuminate conceptual conflict, by structuring discussions so that all students will justify and explain their views and reconsider their ideas in light of new information.

• Teaching to illuminate conceptual conflict will require a change in traditional teaching




methods and teachers themselves (Ames, 1992; Roth, 1990). Conclusions Understanding conceptual change in social studies classrooms requires researchers to take in-depth, micro-level views of student learning. Through long-term engagement in the field, social studies researchers can make important contributions to understanding the process of conceptual change as well as to improving the effectiveness of concept-based social studies instruction. Their personal agendas, interests, cognitive engagement and motivation as well as contextual aspects of the curriculum influenced students’ understandings of human rights. The role of emotional salience in enhancing motivation and cognitive engagement was suggested as an important area for further research. The findings of the study support a comprehensive view of conceptual change in the elementary social studies classroom. While the results are valuable in their illustration of the limitations in student learning when teachers do not challenge student misconceptions, the field also needs studies that analyse successful conceptual change efforts. Research on conceptual change is vital if there is a desire to give students the ability to apply relevant concepts to their future schoolwork and their lives as active members of their communities.

Documents

Type of citizenship education