Assessing Emotional and Social Competence in Primary School and

Assessing Emotional and Social Competence in Primary School

and Early Years Settings: A Review of Approaches, Issues and Instruments

Laurel Edmunds BSc, MSc, PhD, PGCE Research Fellow Unit of Perinatal and Paediatric Epidemiology Department of Child Health, University of Bristol Sarah Stewart-Brown BM BCh, FRCPCH, FFPHM, FRCP, PhD Professor of Public Heath Division of Health in the Community Warwick Medical School, University of Warwick

This project was undertaking when both authors were working at: Health Services Research Unit Institute of Health Sciences University of Oxford Old Road Headington Oxon OX3 7LF

2

All descriptions of assessment frameworks and instruments are subject to the authors' interpretation at the time of writing and do not necessarily reflect those of the Department for Education and Skills.

3

Contents Page

Glossary 4 Executive summary 8

1 Introduction 13

2 Setting the scene 16

3 Methods 23

4 Issues relating to assessment and measurement of emotional competence: the professional context

26

5 Instruments:- 40 Identified in the academic literature 41 Identified by contact with researchers and

practitioners: completed 46

Identified by contact with researchers and practitioners: in development

50

6 Discussion and Conclusions 63 References 68 Appendices Methods 76 Tables of Instruments 82

4

Glossary Assessment: the process of collecting information for the purposes of making decisions about individuals or as part of research to evaluate the impact of interventions. These may involve screening, diagnosing, labelling, placing children in programmes, monitoring of individual children, and evaluating the outcomes of interventions and programmes. Construct: systematic accruing of perceptions relating to the domains of self, with an overlay of cultural and societal relevance and being amenable to assessment. Domains: different aspects or dimensions of the self e.g. social competence, appearance, behaviour etc. Emotional awareness: the ability to recognise one’s own feelings and to differentiate between them. Emotional competence: the ability to understand, manage and express the social and emotional aspects of one’s life in ways that enable the successful management of life tasks such as learning, forming relationships, solving everyday problems, and adapting to the complex demands of growth and development. Emotional intelligence: the ability to perceive accurately, appraise and express emotion; the ability to access and/or generate feelings which facilitate thought; the ability to understand emotion and emotional knowledge; the ability to regulate emotions to promote emotional and intellectual growth. Emotional literacy: the ability to recognise, understand, handle and appropriately express emotions. Emotional regulation: managing one’s own emotions, e.g. soothing self and controlling the way in which anger is expressed. Emotional wellbeing: a holistic state which is present when a range of feelings, among them energy, confidence, enjoyment, happiness, calm and caring, are combined and balanced. Empathy: the ability to be aware of, to understand, and to appreciate the feelings of others. Formative assessment: an assessment undertaken with a view to improving e.g. behaviour or mental health rather than judging it. It can be undertaken as part of a profiling or monitoring exercise, and can show how competencies are developing. Normative scores or norms: these are average scores from large numbers of individuals that are representative of the population as a whole. Positive mental health: Mental health is more than the absence of mental illness. Individuals with good mental health: - develop emotionally, creatively, intellectually

5

and spiritually; initiate, develop and sustain mutually satisfying personal relationships; face problems, resolve them and learn from them; are confident and assertive; are aware of others and empathise with them; use and enjoy solitude; play and have fun; laugh, both at themselves and at the world. Psychometric testing: these are the statistical tests conducted on instruments that assess psychological aspects of behaviour (and many other factors). Psychopathologies: mental disorders. Perceived competence: a general statement of ability across a domain. Instrument reliability: tests to find out if instruments measure what they are intended to measure in a consistent manner. Self-concept: ‘the individual as known to the individual’. A self-description informed by a multitude of the attributes and roles through which an individual judges him/herself in order to make self-esteem evaluations. Self-esteem: generally thought of as a global, relatively stable evaluative construct that reflects the extent to which an individual feels positively towards him/herself. This positiveness is a reflection of the criteria that are central to the value system of the individual, which may or may not be similar to that of the society in which s/he lives. Self-perceptions: a generic term for self-referent statements that are global and specific about the self. Social competence: behaviour, attitudes and understanding that support the development of good relationships and enable children and adults to be successful in tasks involving others. Socially desirable behaviour: behaviours which others usually in authority deem to be socially helpful. Social conformity: behaving according to proscribed social rules or group norms. Social dysfunction: behaving in a manner not acceptable in the host context and culture which may indicate a lack of competence or an underlying psychopathology, or presumably different values and norms to those of the host culture. Social intelligence: the understanding of group dynamics, social status, political relationships, interpersonal activities and leadership. Summative assessment: an assessment undertaken to judge performance or behaviour. Instrument validity: tests to find out if instruments measure what they purport to measure.

6

Instrument Names and Abbreviations AcE Accounting Early for Lifelong Learning BERS Behavioral and Emotional Rating Scale CBCL Child Behaviour Check List CBCL/1½-5 Child Behavior Checklist/1½-5 C-TRF Caregiver-Teacher Report Form CBCL-TRF Child Behaviour Check List – Teachers Report Form CBRS Child Behavior Rating Scale CDP Child Development Project CISS Coping in School Scale CTRF Conners’ Teaching Rating Form DECA Devereux Early Childhood Assessment Program DISCO Diagnostic Interview for Social and Communication Disorders EBDS Emotional and Behavioural Development Scales EBS Emotional Behaviour Scale EDI Early Development Instrument EIPBAS Emotional Instability Prosocial Behavior and Aggression Scales ELA Emotional Literacy Audit ELLI Effective Lifelong Learning Inventory EQ-i:YV Emotional Quotient Inventory FOCAL FOCAL HCSBS Home and Community Social Behavior Scales ICS Interpersonal Competence Scale ITSEA Infant-Toddler Social and Emotional Assessment LIS Learning Involvement Scale PASS Pupil Attitude to Self and Score PBCL Pre-School Behavior Check List PIPPS Penn Interactive Peer Play Scale POMS Process-oriented Monitoring System PSWQ-C Penn State Worry Questionnaire – Children SAT Separation Anxiety Test SCoT Social Competence Test SDQ Strengths and Difficulties Questionnaire SPAI-C Social Phobia and Anxiety Inventory for Children SSBS School Social Behavior Scales SSRS Social Skills Rating Scale STEPS Short Term Education and Pupil Support Talkit Talking Tool Kit renamed Talkit

7

Executive Summary Introduction This review was commissioned by the Department for Education and SkilIs (DfES) . We were asked to identify and review instruments which assess or measure emotional competence in children aged 3-11 years, in three different but related contexts:- early identification, profiling and monitoring. Because emotional competence is so closely related to social competence and because the concepts are so often considered together in the same instrument, this review also covers instruments assessing social competence. We were asked, in addition, to undertake a qualitative study with a purposive sample of teachers, practitioners and researchers, to identify current concerns relating to the assessment and measurement of these concepts. Setting the Scene Our report includes definitions of emotional and social competence and a discussion of the way in which these concepts overlap with related concepts such as emotional literacy and social dysfunction. It also includes a brief overview of the recent development of assessment and measurement frameworks. Methods Our methods were designed to capture as many emotional and social competence assessment frameworks as possible, including those that are currently available and those in development. Searches of the published literature were carried out using six electronic databases from 1990 to 2002 and a wide range of search terms. Notices were posted on the CASEL and Focus Project website notice boards. Contact was made with academics known to be active in the field. All the LEAs in England were contacted, to identify local approaches to assessment. Letters were also sent to the personal, social and health education advisors in all the local education authorities and leaders in Early Years Development and Childcare Partnerships. We aimed to identify instruments appropriate for children aged 3-11 years which were applicable to the general population. We excluded instruments if they were developed with and for groups of children with specific diseases or conditions, or were published in languages other than English. We included all instruments which appeared to assess some aspect of emotional or social competence, regardless of whether they had been designed for this purpose. The instruments we identified were evaluated on the basis of their content, method of application and evidence relating to their reliability and validity and appropriateness for early identification, profiling and monitoring of emotional or social competence. We have not commented on the suitability of instruments for other purposes which might (see above) include the purpose for which they were developed. We carried out a qualitative study by mailing or telephoning a questionnaire (with open questions i.e. respondents were given the opportunity to tell us their opinions) to all those who had responded to our initial request to identify instruments. We also carried out electronic searches for papers discussing measurement of emotional and social competence in general, and searches using the names of common instruments as the search terms. We sought information about the following issues:- the development of instruments, the timing of assessment, the context of assessment,

8

who should carry out assessments, the nature of effective assessment, the barriers to assessment, the significance of the teacher/practitioner, issues for high risk groups, views of parents, teachers, practitioners and children and the purpose of assessment. Responses were received from more than 20 practitioners (some replies were based on the collective response of a group) with a wide range of relevant occupations. Comments are from practitioners who found the time and were inclined to respond and so they may or may not be representative. Issues Relating To Assessment and Measurement of Emotional Competence: The Professional Context We identified a lot of interest in the assessment of emotional and social competence, but also doubts about the extent to which assessment was possible and concerns about the consequences of assessment. Respondents spoke mostly in the context of the assessment of children with identified emotional and behavioural difficulties. In this context formative assessment, which included the voice of the child and that of many other observers, was viewed as particularly important with the aim of helping the child. Summative assessment was viewed by some as judgemental and counterproductive. A small number of respondents talked about assessment in the context of identifying children who could benefit from special help. In this context as in others, time pressure on teachers/practitioners was seen as a problem. The need for teachers to be skilled in assessment was identified as important. Some teachers may be more interested in developing these skills than others. Lack of resources to help children identified as in need of help was another issue raised. A small number of respondents also spoke about the value of monitoring the emotional competence of the whole school to observe the impact of interventions. However, concern was expressed by several respondents about the potential for using the results of competence assessments to rank schools or children. Instruments identified through literature searches and through contact with those working in the field We identified 58 instruments from around the world and classified them by their type and purpose. The first group of 25 instruments were identified primarily by electronic searching of the academic literature. These instruments were on the whole well validated and showed good reliability. Over half of the instruments had been developed primarily to detect antisocial behaviour and most comprised descriptions of behaviours considered pathological by experts. The second and third groups covered the 33 instruments identified through contact with those researching and working in the field, the second dealing with the 23 which were complete, and the third with the 10 in development. The second group included fully developed instruments that have not yet been published. These differ from those in group one in that they all include positive questions or statements and many include aspects of emotional competence. These instruments had been subjected to varying degrees of validation and reliability testing. Some had been developed and tested using a psychometric approach, others using a participative, iterative approach in which the primary concern was the face validity and veracity of the instrument. With regard to the assessment of emotional competence some of the most relevant instruments were still in development using newer approaches, including child participation. We were not able to view all of these instruments and few have yet been subjected to full evaluation.

9

Discussion and Conclusions We found a lot of activity and interest in the assessment of emotional and social competence. Most of the instruments included in the review focused on social competence, but we identified several which assess aspects of emotional competence and are potentially suitable for use in the three contexts screening, profiling and monitoring. The instruments most relevant to emotional competence assessment were applicable to school settings rather than early years. Practitioners had strong views about how assessment might be used. The value of school assessments as part of the development of whole school approaches were not widely understood by teachers and practitioners.

10

1. Introduction The Department for Education and SkilIs (DfES) commissioned the Health Services Research Unit, University of Oxford, to undertake a review of emotional competence assessment frameworks for children aged 3-11 years and a qualitative study to identify concerns of teachers, practitioners and researchers with regard to such assessment. The review was based on searches from the literature base and assessment frameworks identified by contacting other researchers and practitioners in the field. Because the concepts are closely related and often covered in the same instrument we reviewed measures of social as well as emotional competence. We describe the trends in the literature and current thinking in relation to assessment. The qualitative study revealed many concerns related to the assessment of emotional competence. . The 58 instruments identified have been described, and all the findings and issues are brought together in the discussion. This report presents the findings of a project commissioned by the Department for Education and Skills. We were asked to identify approaches to assessment of social and emotional competence and to review instruments developed to measure these concepts, focusing particularly on emotional competence. In doing so we were asked to take account of the published literature, as well as the views of teachers, practitioners, educational psychologists and researchers on the value of making such assessments and measurements. Emotional competence is a relatively new concept and there is still some debate about what it entails. It has clear links, and some overlap, with the more established concept of social competence, and in carrying out this project it has proved very difficult to separate the two. Many of the instruments we identified cover both concepts. The literature illustrates an evolutionary process in the development of understanding of these concepts. Early instruments focused entirely on antisocial behaviour and these evolved into instruments which incorporated the more positive concepts of social competence. Later instruments recognise the emotional underpinning of behaviour, both anti- and pro-social. Now instruments are being developed which acknowledge and try to measure the skill which enables people to use and manage their emotions and be socially competent – the skill of emotional competence. Because much of this literature has been developed by those interested in educational performance, some of the instruments also cover academic competences. Whilst we were undertaking searches for instruments we spoke to both practitioners and researchers about the issues involved in measurement. They provided us with lucid views about what assessment of this type can and cannot be expected to do. It is clear that there are still diverse views about emotional competence, and varied opinions on the practicality and appropriateness of trying to measure children’s development in this regard. It is also clear that assessment and measurement is potentially useful in several different contexts, and that each of these demands something rather different of the instruments. The contexts we identified were: helping teachers/practitioners and others to identify children with poor emotional competence (screening); helping to identify individual children’s emotional strengths and weaknesses in a range of different settings (profiling); helping teachers and practitioners to identify ways to support such children’s emotional development

11

(improving practice); and helping to monitor emotional development, an important component of improving practice (monitoring progress). Most of those we talked to spoke about assessment with individual children in mind, but we also identified interest in whole school approaches. In the latter context assessment might aim to gauge the capacity of schools and settings to foster and develop emotional competence, identify those which are more or less successful in this regard, offer support to the less successful schools and settings and monitor their progress towards improvement. It is clear that emotional competence, albeit under different guises and described using different terminology, is something which is beginning to be of interest to a range of different disciplines, and that people working in many of these disciplines are beginning to tackle the challenge presented by assessment and measurement. The new UK Foundation Stage Profile (QCA, 2002), which replaces baseline assessment with a single exit assessment proposes a number of items reflecting emotional and social development (e.g. forms good relationships with peers, displays a strong sense of self-identity, and is able to express a range of emotions fluently and appropriately) based on best practice, but the validity and reliability of this profile have yet to be measured. Teachers/practitioners are also looking for methods of assessing emotional and social competence beyond the Foundation Stage. There is therefore, a need to identify reliable and valid measures of emotional and social competence that might be suitable for use particularly in early years and primary school settings. Chapter 3 of the report describes the methods we used. Chapter 4 entitled ‘the professional context’ describes the responses and conversations we had with practitioners and researchers relating to their interest in, and concern with, effective and appropriate assessment of children from this perspective. Chapter 5 briefly describes the 58 instruments we identified both from the published literature and from contacting people active in developing research and practice in this area. We reviewed the reliability and validity of each instrument as well as documenting the purposes (screening, profiling etc.) for which it might be useful. These details are set out in a series of tables in Appendix II. We have identified a wealth of interest in this challenging area of research and practice and the main findings and conclusions are set out in a condensed form in the Executive Summary. 1.1 Aims and Objectives

We aimed to identify literature relating to emotional competence assessment and to identify published and unpublished instruments and those in development. We also aimed to identify views relating to emotional competence assessment among those working in this context in England.

The objectives were:

• To search electronically for research describing issues surrounding the measurement of emotional competence.

• To review this evidence and report on published studies from 1990 onwards.

12

• To search electronically for assessment frameworks and tools for measuring emotional competence in children 3-11 yrs and developed since 1990

• To search for assessment frameworks which are not yet in the public domain or were not identifiable in the electronic searches, by contacting those known to be active in research or development in this area.

• To describe the assessment frameworks and critically appraise their content, reliability and validity.

• To contact those working in the field in England to gather their views on the measurement of emotional competence.

• To review issues related to emotional competence measurement and comment on the instruments identified in the light of these issues.

1.2 What the report will not do • Identify and describe every instrument that has been developed. • Identify all the concerns of teachers/practitioners or present the results of a

representative survey. • Describe all the projects that teachers/practitioners and others told us about, in

which they aim to promote emotional and social competence in schools and settings.

13

2. Setting the Scene This section defines the concepts of emotional and social competence and relates them to other concepts such as emotional literacy. It describes the reasons for the current interest in assessment and provides a brief review of its history. It provides definitions of emotional and social competence, describes how these relate to other similar concepts and touches on the difference between social competence, socially desirable behaviour and social conformity. It describes the way in which instruments have evolved from those whose contents were defined by experts with a view to identifying children with problems, to a more child-centred holistic approach, which takes account of social contexts and the impact of the observer. It concludes by describing our approach to the identification and description of assessment methods. 2.1 What is emotional competence? The term ‘emotional competence’ is relatively new and there is still some discussion about its meaning, particularly the way it relates to concepts such as emotional literacy (Sharp & Faupel, 2001) and emotional intelligence (Salovey and Mayer, 1990). Emotional literacy is defined as ‘the ability to recognise, understand, handle and appropriately express emotions’ (Sharp & Faupel, 2001 p. 1). Emotional intelligence is a somewhat broader concept, which includes emotional literacy. It is defined as ‘the ability to perceive accurately, appraise and express emotion; the ability to understand emotions and use emotional knowledge; the ability to access and or generate feelings which facilitate thought (creativity), and the ability to regulate emotions to promote emotional and intellectual growth (Salovey and Sluyter 1997, p10). Emotional regulation includes for example the ability to self soothe and to manage anger. The first two components of emotional intelligence are very similar to emotional literacy, the second two are broader. Elias (Elias et al., 1997) defines emotional competence as ‘ the ability to understand, manage and express the social and emotional aspects of one’s life in ways that enable the successful management of life tasks such as learning, forming relationships, solving everyday problems, and adapting to the complex demands of growth and development’. This definition includes all the attributes of emotional literacy and most of the attributes of emotional intelligence, but places these in the context of relationships and problem solving. Emotional competence and emotional intelligence are therefore concepts with a common core. This common core is emotional literacy. The definition of emotional intelligence extends to cover the contribution which these aspects of human functioning make to creativity and intellectual growth. The definition of emotional competence extends to cover their contribution to relationships and to problem solving. Emotional competence, as defined here, therefore overlaps with social competence (see below). There is consensus among those who have studied these concepts, that emotional literacy, competence and intelligence are important for lifelong achievement, and for the development of emotional and social wellbeing and positive mental health. The latter is a term which has been developed by those working in health promotion to circumvent the problems created by the euphemistic use of the term ‘mental health’ to refer to mental illness services. It covers self confidence and self esteem, resilience, agency and autonomy, the capacity to learn, grow and develop, creative thinking, and trusting, supportive, respectful, interpersonal relationships (Mental

14

Health Foundation website). The areas of overlap with emotional intelligence include the capacity to grow and develop, and creative thinking. Emotional literacy is necessary for the development of supportive, respectful, trustworthy relationships and successful relationships are also part of the definition of emotional competence. Emotional competence is therefore also related to positive mental health. Agency (the belief that one can have an influence on the world), autonomy (the ability to think and act independently of others) and resilience are attributes of positive mental health which are not explicitly covered in definitions of emotional competence, emotional intelligence or emotional literacy, but are necessary for growth and development. Weare has argued cogently that autonomy is important for social competence (Weare, 2000) and Steiner has proposed that emotional literacy leads to emotional wellbeing and positive mental health because it enables individuals to act autonomously (Steiner, 1997). We have construed social competence as behaviour, attitudes and understanding that support the development of good relationships and enable children and adults to be successful in tasks involving others. This definition is concordant with the definition used by Weare (Weare, 2000) in which she describes the three key attributes of social competence as empathy, respect and genuineness. Emotional competence plays an important part in the development of social competence because it enables children and adults to identify and think about their feelings, handle them appropriately and to make a decision about how to behave in the light of both their feelings and their thoughts. For example, an angry child instead of displaying impulsive aggressive behaviour can make the decision to explain what it is that has made him/her angry and request whatever it is that he/she would prefer. This is likely to get a response from others that improves relationships. Aggression, or the other common emotionally incompetent alternative of withdrawal, are likely to result in a deterioration of relationships. Until recently, academics and practitioners have paid more attention to children’s problem behaviour and lack of social competence than to the positive aspects of social competence and the emotional underpinnings of behaviour. Many have focused in particular on socially undesirable behaviour (that which others deem inappropriate in certain settings). Much of the work in this area has been conducted in the USA and has originated in the need to identify children with social and behavioural problems in order that they might be offered specialised help. It is important to make the distinction here between socially competent, socially desirable and socially conformist behaviour. Social desirability often includes an element of social conformity – of not rocking the boat. It may also include behaviours that suit the assessor – children who withdraw rather than complain when they are distressed by something an adult has said or done. Emotional competence, however, does not necessarily lead to socially conformist behaviour. Indeed in some instances it might require non-conformist behaviour – taking an ethical, but unpopular stance for example. It also might encourage children to take issue in a mature way with adult behaviour, when the latter is distressing them. Topping, however, has argued that the concept of social competence includes ‘processing and using the ability to integrate thinking, feeling and behaviour to achieve social tasks and outcomes valued in the host culture and context’ (Topping, 1998). If the host culture does not value dissenting voices, or assertive children, as in the examples given above, such behaviour would fall outside the Topping’s definition of social competence. In the

15

North American literature, particularly in early publications, the concept of social competence is clearly conflated with social conformity and compliance with social norms. These measures of social competence are therefore often concerned with behaviours that benefit adults rather than children (John, 2001; Weare, 2002). Although in certain circumstances – for example in resisting peer pressure to take drugs – parents, teachers, practitioners and society in general value autonomy in children, in others they appear to demand compliance. Conformity in children makes the job of parents, practitioners and teachers easier, but it may be counterproductive in terms of the development of desirable attributes such as positive mental health and good citizenship. We have taken the line that compliance and conformity are not an essential part of social and emotional competence. We have, however, had to work with a literature which has used the term social competence to mean something slightly differently from the way we have defined it here. 2.2 The historical context of assessment Some early papers on the subject of social competence from the 1970s and 1980s assumed that social behaviours, both desirable and undesirable, could be discrete, that is that they were not part of overall behaviour patterns, related to personality traits, nor context specific. Measurement might focus on an undesirable behaviour such as ‘hitting’, observing whether children did or did not hit out in reaction to artificial, socially challenging situations in a laboratory. It did not take into account the fact that hitting is one observable end point of complex emotional/social interactions dictated in part by a particular social situation, not a discrete behaviour to be elicited in isolation from the social context. Instruments attempting to assess behaviours in this way have been shown to lack social validity because they do not represent what happens in the real world of the child (Bierman and Welsh, 2000). Others working in this field have questioned whether antisocial behaviour and social competence are even part of the same continuum (Merrell et al., 2001). Merrell has suggested that it may not be possible to infer levels of antisocial behaviour from instruments measuring social competence. Many early instruments were developed on the basis of observations of experts about what constituted socially desirable/undesirable behaviours. They devised the contents and went on to test their instruments on relevant populations. Many of these instruments proved to have strong psychometric properties (they had good reliability and validity), but their contents limit their usefulness. They were developed from the belief that the observations of experts could be entirely objective. There is now ample evidence that the researchers’ own beliefs and attitudes, which are socially and culturally constructed, have an impact on the contents of such instruments and on how these instruments are used/recorded with individual children. There is also evidence that children’s beliefs and goals impact on their relationships and behaviour but, as yet, these have not been integrated into measures of social competence, described as social functioning by Dweck and John (Dweck, 1991 cited by John, 2001). This evidence is leading researchers to develop instruments which incorporate the child’s voice and recognise that their behaviour may be a reaction to the people who are observing them. Researchers have been aware of the fact that individual characteristics and experiences, with peer groups in particular, are important in the process of adapting

16

to the social environment, facilitating the development of a wide range of social skills (Braza et al., 1993). However, it is only recently that this thinking has begun to inform instrument development. One project currently in progress in Bristol is exploring with 11-12 year-olds how they would evidence doing well at school (Tew, personal communication, May 2002). Tew has discovered that the evidence children use is different from that which teachers/ practitioners would deem appropriate. Children’s evidence was not based on academic achievement. The instrument Tew is developing is an example of one based on the child’s view. The assessment tool developed for the Enable Project (Banks et al., 2001) also incorporates the child’s perspective. This instrument has undergone several iterative stages over the past ten years evolving in response to the views of those using it. It enables the practitioners, researchers and children using the instrument to voice their opinions about which behaviours are relevant to them. Prior to the 1990s, many instruments were developed with the aim of identifying children with specific conditions and pathologies and their content focused on the negative. In response to concerns about the need to recognise and build on strengths there has recently been a move to develop instruments which also assess positive generic attributes and strengths. The latter are more appropriate for work with ‘normal’ populations. This change has been accompanied by changes in terminology. In the early ’90s, authors were interested in social behaviours and behavioural problems, i.e. observing behaviours and inferring social problems, whereas latterly authors are more likely to focus on competences (Mitchell-Copeland et al., 1997; Fantuzzo et al., 2001). This approach has enabled the development of instruments that aim to work in partnership with children to help them improve their emotional and social competences. Such instruments enable formative assessments – those whose primary purpose is to support development – as opposed to summative assessments whose primary purpose is judgement about whether a child is competent or incompetent. They are useful for profiling children and for supporting improvements in practice. In spite of these advances, many instruments in current use still require raters to score children on the basis of behaviours which are deemed by experts to be abnormal, with the primary goal of identifying problem children. These do not require the raters to reflect on the social context in which the behaviour is being observed nor the child’s view. Such instruments tend to be more evident in the context of screening to identify problem children. 2.3 Approaches to assessment Instruments have therefore been used in different ways. They all share the aim of improving outcomes for children, but support rather different approaches to doing so. Broadly speaking they fall into three categories – early identification (screening), profiling (perspectives from different sources about the same child or school) and monitoring. Screening instruments need to be able to separate children who are likely to need special support or intervention from those who do not. As they need to help practitioners make a judgement, they are summative. Because all children will be screened, these instruments also need to be appropriate for mainstream children and quick to administer. They also need to have high sensitivity (which means they will miss few children) and specificity (which means that they will not incorrectly judge

17

mainstream children and those with special needs). They can be completed by a range of different observers including the children themselves. However the fact that they work well as a screening instrument with one type of observer (e.g. teacher) does not mean that they will work well with another (e.g. child). Independent evaluations need to be undertaken of their reliability with each reporter. Screening is only worthwhile if the problem sought is not obvious to a casual observer. It is also only worthwhile if an effective intervention is available to children identified as having problems. Although the latter are characteristics of screening programmes not screening instruments, they are important in deciding whether screening is likely to be worthwhile. Instruments suitable for profiling need to be able to describe a child from many different perspectives in such a way that they enable both adults and children to identify what might be done to improve their emotional and social competence. Profiling is therefore formative and it requires different observers. It provides an all round picture of a child and suggests approaches and interventions that would support the development of the child’s emotional and social competence. As it is usually used with children whose development is less than optimal, the instruments do not necessarily need to be appropriate for all children. Instruments suitable for monitoring need to be able to assess change over time. They need, like profiling and screening instruments, to be valid and reliable, but they also need to be able to detect change accurately. So they need to help a practitioner decide whether what they are doing with a child is resulting in improvement. All three types of instrument can be used for individual children or for groups of children in a school, classroom or early years setting. In the latter case the aim would be to identify schools or settings in need of special support (screening), or to identify the strengths and weaknesses of a class and suggest ways of supporting class development (profiling). Monitoring is often carried out at the group level and in this case precision is not so vital. Instruments which are not very accurate at individual level can sometimes perform as very reliable indicators of progress at group level. 2.4 Our approach We have taken a broad approach to identifying instruments, aiming to examine those developed by different disciplines in different contexts. We have used the authors’ terminology when describing the instruments, but also tried to clarify what the instruments actually measure. We have documented the purposes for which the instrument was developed, the age group for which it is recommended, the context for its use, and who it is recommended should carry out the assessment. In reviewing the literature, and in speaking to practitioners and researchers, we have also aimed to establish the purpose of assessments, what is viewed as effective assessment, what the barriers to assessment are, the significance of the assessor in the process of assessment, any specific issues relevant to high risk groups (those who are likely to be at higher risk of emotional or social incompetence e.g. children in families living in social deprivation and groups, those from some ethnic minority groups) and the views of parents, teachers, other practitioners and children on assessment.

18

These background details are vital to the successful application of instruments. However, it would seem that this information is rarely reported in the academic literature relating to instrument development. In their review of six rating scales, Demaray et al. (1995) comment that no background information such as the above was available for instruments reviewed. This is partly due to limitation of space in journals, but also due to the publication biases. From a scientific point of view objective details such as the psychometric properties (validity and reliability) of instruments seem to be regarded as of pre-eminent importance and this background information of less interest. Practical issues associated with the use of instruments are more likely to be reported by authors who have used the instruments as part of another study. For some of the instruments we identified from the peer-reviewed literature, we have conducted secondary literature searches in an attempt to answer these questions.

19

3. Methods This section describes the methods used to gather the information presented in this report. It describes the electronic searches of the published literature, our approach to contacting academics who are active in research in this area at the moment and methods of contacting practitioners working in this field in England. 3.1 Identification and critical appraisal of instruments and assessment

frameworks The project aimed to identify as many approaches to, and instruments for, assessing and measuring emotional and social competence as possible, including those that are currently available and those in development in 2002. Instruments were identified through a variety of methods. Literature reviews were conducted by searching six electronic databases from 1990 to 2002. These databases were: ERIC Education research articles PsychInfo Journal Articles and Chapter/Books (Psychological research) Sociofile Social Science Citation Index Health Star Journal articles (Health research) Medline Journal articles (Medical research) Embase Journal articles (Medical research) The search strategy was broad, aiming to identify instruments covering emotional literacy, emotional intelligence, social competence and socially desirable and problem behaviour, as well as emotional competence. Search terms included: emotional competence, social competence, emotional literacy, emotional intelligence, emotional awareness, emotional regulation, interpersonal sensitivity, emotional wellbeing, behaviour and mental health. All of the above were searched in turn with each of the following terms: assessment, instruments, measures and tools. A notice was posted on the Centre for the Advancement of Social and Emotional Learning (CASEL) website notice board with regard to our search for emotional competence assessment frameworks, and the study was flagged up on the Focus Project (promoting effective practice in child and adolescent mental health) website. This UK project was launched in 1997 and aims to promote clinical and organisational effectiveness in child and adolescent mental health services, with an emphasis on incorporating evidence-based research into everyday practice. One of their objectives is to collate and disseminate up-to-date knowledge of effective practice, both in terms of health and social care interventions and of the organisations which deliver care. The child database of the Oxford Outcomes Project was also searched. This database is part of a larger one documenting the development and application of questionnaires, interview schedules and rating scales that measure states of health and illness from the patient’s perspective. These instruments are completed by patients to provide a measure of their experiences and concerns in relation to illness, health status and quality of life. The database is funded by the Department of Health to support health services research, by enabling researchers to identify valid, reliable and appropriate measures for use in their studies. It currently contains 310 records of outcome measures relevant to child health and wellbeing including emotional and social wellbeing.

20

Contact was made with academics known to be active in the field. All the LEAs in England were written to, to identify approaches to emotional competence assessment in current use. Letters were also sent to the Personal Social and Health Education (PSHE) advisors in all the local education authorities and lead officers in Early Years Development and Childcare Partnerships. The criteria for selecting instruments for inclusion in the review were:

• for use with children aged 3-11 years in early years settings and schools, • applicable to the general population aged 3-11 years (or where instruments

were developed for specific populations, involved mainstream or ‘control’ children as well).

Exclusion criteria:

• instruments developed with and for disease-specific populations, • instruments developed or published in languages other than English, • instruments and scientific papers published before 1990.

In choosing to restrict our searches to 1990 onwards, we were aware that this would exclude well-established instruments measuring some aspects of social competence (e.g. the Waksman Social Skills Rating Scale, and the School Social Skills Rating Scale (cited in Demaray et al., 1995). Our rationale for the restriction was the lack of mention in the literature prior to this date of the concepts which were the focus of the review – emotional competence, emotional literacy and emotional intelligence. We have included all the instruments we identified which touch in any way on these concepts, even though many of them focus on social competence, abnormal behaviours and psychopathologies (John, 2001). It is unlikely, given the state of development of electronic coding of this literature, that we have identified all potentially relevant instruments. We have, however, identified a wide range of instruments and believe that those we found provide a good overview of those that are currently available. The following information was extracted about each instrument:

• type and content of the instrument, • characteristics of the population on whom it had been tested, • method of application, • descriptions of tests for validity and reliability of the instruments.

Instruments were evaluated on the basis of their content, method of application and the evidence relating to their reliability and validity in relation to the assessment of emotional and social competence. These results are described in the tables in Appendix II. Some of the instruments we included were designed primarily to assess competences/pathologies other than emotional/social competence as defined in this report. Our appraisal of these instruments relates to their suitability for assessing emotional or social competence and not to their suitability for their intended purpose.

21

3.2 Identification of issues and concerns relating to emotional competence

assessment We were asked, in addition to identifying instruments, to identify published literature and current opinion primarily in England in relation to key concerns with respect to the measurement of emotional competence. The issues were:

• what is the purpose of assessment? • when should children be assessed and in what context? • who should carry out the assessment? • what is effective assessment? • what are the barriers to assessment? • what is the significance of the teacher? • are there any special issues for use with high risk groups? • what are the views of parents, teachers/practitioners and children?

We addressed this request in three ways. In our electronic searches of the literature, we identified and obtained papers whose titles and abstracts suggested that they discussed these issues. We also conducted a secondary search of the literature focusing on five commonly used instruments covering different aspects of social and emotional competence (Interpersonal Competence Scale, ICS; Infant and Toddler Social and Emotional Assessment, ITSEA; Strengths and Difficulties Questionnaire, SDQ; Home and Community Social Behaviour Scales, HCSBS; Child Behaviour Checklist, CBCL) to identify studies relating to instrument use rather than development and validation. This literature has contributed to our discussion of the issues throughout the report and is referenced as appropriate. Findings of particular relevance to individual instruments are reported in Section 5 where the instruments are discussed. In addition, we developed a brief semi-structured questionnaire , which covered these questions. We mailed, emailed or telephoned the questionnaire to all those who had responded to our initial written communication to researchers, LEAs, Personal and Social Health Education (PSHE) and Early Years Development and Childcare Partnerships. As the questionnaire was given only to those who had already heard about the project and expressed an interest in what we were doing, it was preceded only by a brief preamble to explain why we were requesting the information. The responses were analysed and grouped into themes under each of the questions or comments. Because the results of this initiative are important in terms of setting the scene for the presentation of instruments, we present them first.

22

4. Issues Relating To Assessment And Measurement Of Emotional Competence: The Professional Context

This section describes the responses of the practitioners and researchers who responded to our survey asking for their views on key issues in emotional competence assessment, such as whether such assessment should be developed, what the purpose of assessment is, who should carry it out, and what the barriers are. It also includes the views of researchers who have published on the topic. We identified a lot of interest in this area but also some doubts about the extent to which assessment was possible and what the consequences of assessment might be. Respondents spoke mostly in the context of assessment of children with identified emotional and behavioural difficulties. In this context, formative assessment, which included the voice of the child and many other observers, was viewed as very important with the aim of helping the child. Summative assessment was viewed by some as judgemental and counterproductive compared with formative assessments. A small number of respondents talked about assessment in the context of identifying children who could benefit from special help. In this context as in others, time pressure on teachers/practitioners was seen as a problem. The need for teachers to develop skills in this area was seen as important. Lack of resources to help children identified was another concern. A small number of respondents also spoke about the value of monitoring the emotional competence of the whole school to observe the impact of interventions. Several respondents expressed concern about the potential for assessment to be used to rank schools or children. We posted nine questionnaires, sent out 44 by e-mail and conducted five interviews by telephone. We received 23 replies. Respondents included teachers, PSHE advisors, EYDCP managers, personnel from a national programme to improve self-esteem and resilience in primary school children, a primary schools drugs advisor, researchers and a group of educational psychologists. Two respondents found themselves unable to answer the questions, suggesting that responses needed to be context-specific (educational psychologist, EYDCP manager). Both from this exercise and from the letters we sent out to LEAs and PSHE advisors, we gauge that there is a lot of interest in the measurement and development of emotional competence amongst those working in education in England. However, it is important also to note that the views expressed are from a motivated sample with the time and inclination to respond. They cannot be assumed to represent the views of all those working in primary and early years education in England, but similarity of responses may be indicative of a range of some widely held views. Because all those who responded have an interest in the area and are themselves grappling with the issues at the moment, they are, however, likely to represent an informed view. We have grouped responses under the headings listed above (see 3.2), retaining each respondent’s choice of headings as much as possible. This gives an accurate reflection of what they thought was relevant to the topic, but occasionally we have reassigned comments to improve clarity of interpretation. All respondents were from England unless otherwise indicated. We have identified them by their occupation or discipline.

Deleted:

Deleted:

23

4.1 Development of assessment frameworks Only researchers chose to comment on this issue and the responses, perhaps predictably, were positive – ‘emotional and social competencies are central to effective learning. Combining assessment for personal development with assessment for learning may be the best tool we have for raising achievement!’ (researcher). One thought that in addition to individual assessment, frameworks should be developed to assess ‘group dynamics and interaction’ as these ‘… promote positive relationships’. Some reported a need to develop instruments that were socially constructed, i.e. involved the participants in their context and/or developed assessment from the child’s perspective. The national programme practitioner said ‘We are aware of a number of different approaches in different authorities and by different organisations. A standardised approach would obviously provide more comparative data, but would not necessarily meet everybody's requirements’. 4.2 When and in what context should children be assessed? Most respondents agreed that baseline assessment at entry to school was important, but also considered age-appropriate assessment to be of continuing importance at older ages. One of the researchers suggested, ‘abilities such as empathy are more appropriately assessed at Key Stage 2’. While one respondent suggested that ‘It is perfectly possible to do an observational assessment at any age’, she also said assessment ‘in the early years and at primary age would yield information about social skills’. Most respondents said ‘as early as possible on entry to school or nursery’. Caveats were offered about the context, suggesting that children should be assessed ‘in play context as the child goes about daily tasks … not isolated assessment tasks but structured part of daily activity’ (principal educational psychologist). Other respondents mentioned a range of settings in which the baseline and later assessments could take place. These included child development centres, special school nurseries and mainstream pre-school settings. The view was expressed that assessment ‘should not in any circumstances be about judgement or for normative purposes’ (PSHE advisor); rather ‘it should be formative assessment – i.e. encouraging children’s own self-awareness and personal development. It should be relatively informal and should not be associated with any one subject area’ (researcher). A primary school drug education advisor interested in assessment to inform planning suggested ‘a formative assessment … to be made one half- term before the block of learning begins. [it] should be related specifically to the desired outcomes for the following half term and should inform planning, teaching and learning and … summative assessment’. Responses tended to be specific to the context in which the respondent was working. School practitioners see children most days during term time, whereas educational psychologists see children for specific reasons on specific occasions. A teacher will see children in their class on a daily basis during the school term, whereas specialist practitioners only see a child when there is a cause for concern. This difference was illustrated by one of the Special Needs co-ordinators, ‘It depends on how formal the assessment is. Children are assessed all the time, but in an informal way in school. When you do the baseline assessment at entry, children respond in one of two ways, they either shut down or go over the top. We carry out a fuller one at six months and

Formatted: Font: Italic














Deleted: ,

24

then again at a year or when they leave. I think it’s dangerous to do too much formal assessment with children when they are too young’. Responses indicate that most people interpreted the question in the context of supporting children with identified behavioural problems, not in relation to the development of emotional competence in the general population of children or the identification of children with such problems in schools or early years settings. When a child with behavioural problems is assessed depends on Local Education Authorities’ processes and the length of time it takes to gather information about a child from other sources such as social and health services. School practitioners and educational psychologists both highlighted the need for a ‘clear cross-service approach between health and education. Health visitors have a lot of useful information that does not always reach the school or nursery’. In contrast, the published literature was more concerned with instruments which were appropriate for mainstream populations of children. While most researchers suggested an age range over which their instruments were applicable, few were prescriptive as to exactly when they should be used. One of the US researchers pointed out that a benefit of a standardised instrument is that it is ‘intended for all children’ and can be used to ‘obtain pre-post-test ratings’. This respondent also stipulated that ‘raters must know the child for a minimum of four weeks before doing an assessment’ using the instrument with which she was associated. 4.3 Who should carry out the assessments? One of the main themes to emerge from the responses to this question was that child self-report was important and should be included whenever possible. Some researchers working with older children reported that they were developing instruments specifically to give children the opportunity to self-report. One had called her instrument a ‘Talkit’. With regard to other raters, most respondents suggested adults who knew the children and possibly peers were the appropriate sources of information; ‘adults who are in regular contact with the child, i.e. health visitors, teachers and teaching assistants’ (PSHE co-ordinator) or alternatively, ‘the individuals who actually rate the child are teachers and parents who are with children on a daily basis’ (US researcher). Some respondents felt that it was important for children ‘to have formed a relationship and basic trust with the adult observing them’ (principal educational psychologist). A school practitioner suggested; ‘this should be self-assessment, teacher observations and maybe peer assessment. It should be based within the educational context’ (senior PSHE advisor). Researchers tended to put ‘children with their teachers’ as their main choices. The nature and severity of the child’s problems influenced responses. ‘If the child is considered a single agency referral i.e. not severe and complex and therefore not falling under the remit of the Joint Agency Team, then usually education support staff or speech and language therapists will be asked to provide the assessment’ (inclusion co-ordinator). The range of professionals mentioned that might be involved in assessment included pre school advisory teachers, portage advisory staff, speech and language therapists, occupational therapists, physiotherapists, sensory impairment advisory staff and educational psychologists.












25

A response from an inclusion co-ordinator highlighted the influence of the setting on the assessment and the potential influence that teachers/practitioners have on the process: ‘… the settings themselves have a key role to play. They [teachers] don’t always seem to accept responsibility for this, for a variety of reasons’. One of the principal educational psychologists stated: ‘Emphasis … should be placed on the development of cross service work. Assessment should become part of accepted good practice rather than an “add-on”’. One criticism raised about specialist assessment was that ‘the child often has lots of adults and equipment around them and so don’t behave in their usual way’. The respondent would have preferred a method of assessment that provided an ‘everyday picture’ of the child so more help could be given (head teacher, Special Needs school). The research literature reflected many of the views expressed by the practitioners and researchers we spoke to in this study and confirmed that it matters who reports. Martens (1993) suggested that: ‘for behavioural rating scales and checklists, one of the most important systematic sources of variance leading to non-random fluctuations in test score, is the informant’ (p. 310). For example, the person carrying out the assessment may have prior judgements about particular behaviours that differ from those of other raters. These judgements are likely to bias all the ratings that person makes in the same way, (hence systematic). Although measures based on instruments completed by adults are relatively easy to obtain and are much used, they are influenced by ‘a variety of conceptually independent adult-valued variables’ (Braza et al., 1993, p.146). These tend to emphasise positive attitudes and socially desirable behaviours. Therefore any instruments that are reliant upon observers are inherently subjective, being open to error and bias. These unsystematic sources of variance can be countered somewhat, but not entirely, by using psychometrically sound instruments. Demaray et al. (1995) suggest that any single assessment is bound to be limited and that best practice demands multi-sources, multi-situations and multi-methods. 4.3 a Parents Respondents highlighted the reliability of parental contributions to assessment ‘people who know children best, key workers and parents’. However the Head of a special school added ‘but wouldn’t rely on them [the parents]’. She did not qualify this statement. One of the principal educational psychologists took a different perspective and made this point with respect to ‘relationships between home and school – trust and respect are of paramount importance (and have to be established) before parents will share crucial information about their child, particularly if they feel they are [labelled as] not “good enough” parents’. The national programme personnel also pointed out ‘teachers are the primary source of information for the assessment, but we would also advocate that parents, children and other professionals who know the children be involved in any overview – there is evidence that a parent's opinion of their child’s needs can be very different from that of professionals’. Both parents’ and teachers’ opinions of a child may also reflect their needs as well as those of the child. An unbiased picture is most likely to be gained by gathering data from more than one source. Some issues relating to parents as raters were covered in the literature. In a study using the Strengths and Difficulties Questionnaire (SDQ) with 2,000 primary school











26

children in Singapore, mothers returned higher scores than fathers (Bibou-Nakou et al., 2001). A Greek study using the same instrument showed that parents tended to rate their children as showing more emotional problems, and teachers more conduct disorders (Bibou-Nakou et al., 2001). Depressed mothers rate their children more negatively (Fombonne et al., 2001). These differences may be attributable to differences in the way the child behaves in different contexts or to differences in the opinions of the observers. Despite this, Carter et al. (1999) in a laboratory-based study has shown that mothers can provide competent ratings of their child’s problem behaviours as early as one year of age (Carter et al., 1999). 4.3 b Teachers/Practitioners The respondents made few comments about teachers/practitioners specifically, but this was a topic that was covered in other questions and by the literature on assessment. One respondent offered the following: ‘This requires skills in the teacher akin to those used in counselling. They have to remain non-judgemental and respectful of the child’s current position. They have to provide the emotional “space” for the child to reflect meaningfully on their feelings and actions. Some teachers are skilled listeners who can create the required emotional climate, but many are not. Most could learn the skills, the question is whether they would want to’ (practitioner/researcher). ‘To introduce emotional literacy as a mainstream component of education will require a philosophical change from didactic teacher to facilitator of learning. This might be a smallish step for primary teachers to make, but a large one for many secondary teachers’ (advisor/researcher). The academic literature raised similar issues about teachers/practitioners as raters to those it raised about parents. Merrell (1993) reported that teachers make efficient and effective reporters because they see children on a regular basis over a period of time in situations in an area not observed by other adults. Bierman and Welsh (2000) felt that teachers and peers are better placed than parents to comment on children’s behaviour in peer relationships. Teachers’/practitioners’ judgements, however, like those of parents, are subjective. Behaviours that may be the norm (however undesirable) in one school may be exceptional in another. Individual children may become labelled as troublemakers throughout a school. Systematic variation in scores both by school and by teacher can be demonstrated (Bierman and Welsh, 2000; Harris et al., 1993). There is evidence that variability between teachers’ and peer assessments increases as children get older (Fombonne et al., 2001). School contexts can be very different and competences that are expressed in class can be very different from those in free play indoors, in the playground or in games lessons (Harris, 1979, cited in Bierman & Welsh, 2000). 4.3 c Peers Peers were thought by researchers in particular to be a useful source of information in assessment, but as Braza (1993) pointed out, asking peers to rate their contemporaries often provides a reflection of their popularity rather than their social competence. Also interactions between two children need to be viewed from the perspectives of each child. For example if one child hits the other as part of a “rough



Deleted: ‘

27

and tumble”, the response of the second child is important to make a judgement, i.e. the second child could laugh as part of the play or get upset because it was inappropriate and painful (Bierman and Welsh, 2000). 4.4 What is effective assessment and is it possible in the current climate of

quantifiable, objective measurement? Responses to this question were mixed with many respondents questioning whether objective measurement was desirable or possible and those associated with instrument development championing it. Nearly all the English respondents were apprehensive about summative assessment and preferred formative assessment. The latter is usually based on multi-dimensional instruments which include the perspectives of a range of different people and employ more than one method. This type of assessment is carried out with the intention of identifying areas of development where help is needed. Other respondents, particularly those developing instruments, believed in quantifiable and objective measures where a single score is derived from an instrument. ‘Effective assessment is … assessment which is valid and reliable. We would contend that for emotional competency the process (formative assessment) is the key not the outcome (summative assessment). We do not feel that there could be a quantifiable, objective measurement although there are useful ways of assisting a child to work through the process’ (educational psychologists). This was reiterated by two of the researchers: ‘effective assessment needs to be conducted over time, not on a “one off test”. It does not lend itself to “tick box” type assessments where children are categorised, it is far too complex for that’; and: ‘I do not think that this area of development is open to quantifiable/objective measures. It involves personal subjective evaluations and motivational issues which are often only accessible to the person themselves, particularly as they grow older. In my personal view, part of the growth and development is wrapped up in being asked to make subjective assessments of performance in different social and emotional situations and seeing if the repertoire of strategies can be increased and tackle it differently next time. What purpose does an objective measure serve? It does not serve the best interests of the growing and developing person,’ (researcher). The national programme practitioner thought that ‘effective assessment has to be a measurement that is appropriate to the audience and context … the results of which can provide adequate comparable data. … It should be used as indicative only and used with sensitivity. Ultimately, it should be possible to build up a critical mass of information to produce meaningful statistics’. Some respondents from England acknowledged the climate of objective measurement and its potential uses and harms. A senior PSHE advisor said: ‘if we are to value the development of these skills then using formative assessment is important’. One of the principal educational psychologists thought assessment should be proactive and used in the context of screening. ‘It highlights children “at risk” or particularly vulnerable and aims to develop strategies to support them before the issues impact too greatly on overall achievement, including academic performance. Therefore, it is of crucial importance that it does not require children to “fail” before support is offered’. Another caveat came from one of the researchers: ‘Scores are not a good idea, but a profile, or portfolio, or learning narrative which includes celebration of achievement as well as self directed new goals for development may be useful’









Deleted: ’

28

(researcher). Another issue was raised by one of the researchers about using the ‘tick box’ approach. ‘Pupils will often give what they perceive to be the correct answer rather than what they would actually do’. The group of special needs co-ordinators had very clear views about what assessment ought and ought not to be. They considered assessment should include a ‘setting that has information about the child’s learning styles which allows them to differentiate… [where] there is advice and support on suitable resources; [where] parent/carers are involved and understand and support the process; [where] the child has very clear easily measurable individual targets that allow success for that individual to be seen and measured by all concerned including the child; [where] the individual setting has confidence to manage and teach the child and the child is happy, as are parent/carers’. They were also clear about what assessment was not: ‘jumping through hoops to [complete] a statement; a single-minded route to a diagnosis/disease label which then generates no other support help, confidence, or success for the child; an opportunity to say that an individual child is “beyond” the help of any particular setting; an opportunity to “label” an individual child and therefore limit their ability to achieve, in whatever field’. The literature searched identified papers cautioning that ‘performance based methods of assessment need to reflect the complexity and range of naturally occurring interactions in children’. These authors were concerned about interpretation of behaviour. For example, a child who prefers constructive object-oriented play may appear to be socially withdrawn, but not be socially dysfunctional. 4.5 What are the barriers to assessment? Both practical and philosophical barriers were identified in response to this question. The practical barriers included time, which was an issue for many respondents, workload and covering the curriculum. ‘Often children must be observed in a variety of situations before an accurate picture emerges of their needs. The time and cost implications are therefore considerable’. Some of these comments were qualified with statements such as ‘but assessment is an integral part of good teaching so if used wisely there should be no barriers. There would be immense difficulties in my view if the assessment were used to label and to judge the abilities of individual children – one against another’ (senior PSHE advisor). One of the practitioners gave a ‘real world’ response. ‘Assessing emotional competence would be valuable but needs enough time. Assessment is often seen/treated as a bolt-on and there isn’t any funding for it. If you carry out baseline assessment properly, it’s very time consuming if it is to give a fair and meaningful portrait of the child. But people cut corners because of number of children and so it doesn’t get done accurately, with the result that teachers think it is a waste of time. There are considerable dangers with the climate of objective measurement whatever the area of assessment but more so with the “soft skills”. The thought that we might give young people an EQ score as well as a SATS [Standard Assessment Tests (national tests)] score as they leave school makes me shudder’ (head teacher, special needs school).










29

Other practical barriers that emerged included: ‘a lack of responsibility within the school for inclusion so nothing happens while [the school] waits for outside agencies to take responsibility; a lack of liaison between the professionals involved; looking at assessment as an end [point] rather than using it to inform future planning; lack of experience on the part of teachers; a lack of training/expertise and support [for teachers/practitioners]; confidence to change [teacher/practitioner] attitudes and practices; and willingness to assess children for purposes other than the child’s wellbeing’ (senior PSHE advisor) [i.e. teachers will conduct assessments for the child’s benefit but not necessarily for other reasons such as comparisons between schools or settings]. Other barriers reported included ‘the lack of sufficient resources to undertake an assessment and lack of the administrative support necessary for involving children and parents’ (national programme practitioner). Assessment might be seen as just ‘more paperwork’ for teachers/practitioners, which would be self-defeating. The group of educational psychologists expressed their experiences and thoughts on barriers. ‘For schools we believe it is a lack of understanding/acceptance of the importance of this area in relation to learning and achievement. We feel that it is good that emotional competency/literacy is a subject for discussion in education and hope that schools will take on the need to understand where the child is “at” in order to help them learn effectively’ One of the advisor/researchers described potential barriers outside the school. ‘If the assessment takes the form of an objective, summative measurement, then there could be objections from various quarters, including parents. If the child is socially unskilled or emotionally uncontrolled, who are we assessing? Arguably in the case of a young child, we are assessing the parents! They might object!’ Another principal educational psychologist included a potential barrier for the child. ‘For children themselves the main barrier is that they may be referred by staff for group work to overcome low self-esteem but actually perceive themselves “well” based on their reference groups.’ Also as one of the researchers pointed out ‘focusing on measurable outputs may be accompanied by a negative impact on self esteem’. None of the respondents mentioned a lack of a suitable instrument as a primary barrier. Respondents who were developing instruments advocated theirs because they were ‘easy to use and time efficient’ (US researcher). These respondents also raised some provisos: ‘the instrument is an adult’s interpretation of a child and it is just a sample of behaviour.’ For instruments involving child self-report, concern was raised about language: ‘the child may get it wrong because s/he may not understand or s/he understands the concepts but can’t demonstrate it’ (head teacher, special school). 4.6 What is the significance of teacher (assessor) skill, judgement and

evaluation in assessment? What training is required? The consensus view was that ‘teacher professionalism and vision is central’ (researcher) and that teachers/practitioners had to be emotionally competent themselves. This was illustrated by the following: ‘the skills required are those of a













Deleted: .

30

good teacher. If we are going to encourage teachers to actively support the development of the Behaviour Curriculum Skills (Social and Emotional Competencies) then they will need to understand what these skills are and how to teach them. With reflection, the ability to assess the skills will inevitably follow’ (senior PSHE advisor). The national programme practitioner proposed their instrument circumvented any teacher inconsistencies: ‘we are primarily looking for comparable information within one peer group, so a certain lack of objectivity by the teacher can be accommodated, particularly if other parties are able to contribute to any assessment’. Phrases such as ‘very significant’, ‘critical’ and ‘vital’ were frequently used in relation to teachers’/practitioners’ skills as assessors, but with the proviso that ‘it should not be seen out of context of what parent/carers have to offer about their particular child’. The issue of training also came up from two perspectives. Firstly, the educational psychologists thought there was a need to ‘understand what being emotionally competent means and that it’s not just about administering checklists’. Secondly, there was the issue of training teachers/practitioners to put ‘good assessment processes in place for all children first as well as good working systems with parent/carers. Making timetabling opportunities for observation and assessment and making it a priority, as well as doing something with it when it’s done’ (inclusion co-ordinator). A researcher suggested, ‘basic experiential training in emotional literacy should be part of initial teacher training and continuing professional development. Teacher judgement and the manner in which that judgement is made is critical. A lot can be talked and modelled in relation to this domain and thus teachers themselves need to be competent and able to model appropriate strategies and relationships’ (researcher). ‘Skilling teachers/practitioners in this area can have a huge preventative effect’ (researcher). One of the US researchers highlighted the advantages of the instrument with which she was involved. One of the main advantages was that no training was required to complete the instrument (DECA), but training was needed to implement the accompanying DECA programme. The only requirement for the instrument was for the adult rater to have known the child for four weeks. 4.7 Are there separate issues for different high-risk groups of children, e.g.

social class, SEN, ethnicity, gender? The primary response to this question was that children need to be considered as individuals and so the ‘high risk’ aspect of their backgrounds was of secondary importance. ‘[This is an] important but not [a] separate issue[s] if all children are understood in their personal context’ (principal educational psychologist). One of the US researchers pointed out that norms are not available for different groups of children - that is ‘high risk’ and ‘not high risk’. The only different norms that have been published are those relating to parents and teachers as observers. This means that it is possible to assess whether a ‘high risk’ child is different from the normal population, but not whether she/he is different from other children in the same ‘high risk’ group. ‘

















Deleted: w’

31

Special educational needs (SEN) were not seen as a particular problem (senior PSHE advisor). One reason for this may reflect a new approach to thinking about SEN. One newly renamed Inclusion Co-ordinator (who had been a senior SENCO), was at the forefront of a countywide shift in approach: ‘get inclusive systems in place for all children and only then call in outside support agencies if these systems are not sufficient to inform planned success for an individual’. Cultural and language differences were flagged up as an assessment issue (researcher). A PSHE advisor suggested taking a broad approach to assessment to accommodate ethnic and religious differences and ‘work with the broader school community to come up with a framework’. Gender came up as an assessor issue. Many of the practitioners working with young children are women and one senior PSHE advisor suggested that this may affect how children were assessed: ‘There are also gender issues and concern that this might be a “female” approach to life’. Early years settings and primary schools tend to be populated by females which may favour girls in terms of teaching and learning experiences. Some respondents were concerned about brain research showing developmental differences between boys and girls. It is ‘important to consider boys’ readiness for school and [we] need to develop their understanding of emotion and feelings’ (principal educational psychologist). The suggestion here is that children may need to be assessed against gender specific norms, which take into account the developmental lag in boys compared with girls, and that most boys are likely to need additional support to develop emotional competence and school readiness. Other comments included the need for further research into what are the risk and resilience factors and the ‘need to look beyond the traditionally accepted “high risk” groups, as many children of professional parents are now at risk’ (principal educational psychologist). Support for parents together with community capacity building was seen as important to effect sustainable change in disadvantaged areas. Findings from the literature with respect to high-risk groups indicated that social behaviours need to be interpreted against a background of the socially constructed environment which can vary from one culture to another. Eye contact is an example of behaviour regarded an important component of social competence in Caucasian cultures, and as an inappropriate behaviour for girls in certain Eastern cultures. As Fantuzzo et al. (1995) point out, instruments with norms that have been established with white middle-class populations run the risk of classifying non-mainstream children as deviant rather than different. This is particularly pertinent as ethnic mix changes over time and numbers of children from ethnic minority populations grow (Fantuzzo et al., 1995). Saarni (1997) raises a similar issue. By mid-childhood children are incorporating others’ behaviours towards themselves into their self-system (which is intimately linked with emotional experiences in different environments). Children’s first experiences of society outside the home are likely to take place in early years or school settings. If groups are comprised of children from different cultures, whose first language may not be English, assumptions about the appropriateness of social interactions may present a challenge. This is a situation that practitioners in many early years settings and schools are coping with on a daily basis.






32

4.8 Views of schools/teachers/practitioners on assessment Some respondents questioned how seriously the results of assessments were taken, particularly when they were not allocated time to do it properly, or when checklists were issued with little background information on how they were going to be used. As one teacher said ‘the danger of asking teachers to fill in another assessment form that just appears is that if it doesn’t look like it’s being treated seriously, they won’t treat it seriously’ (special needs teacher). It would seem however that it is possible to introduce assessment without these problems. One of the researchers commented that the teachers she had worked with thought that the instrument she was piloting was a helpful way of tracking change, and that the process had not added to their workload. The view from outside the classroom was typified by this response: ‘In my experience teachers do not seem to object to the idea of assessment but are unwilling to have another type of assessment to do’ (senior PSHE advisor). Another response covered this point and the issue of teacher emotional/social competency. ‘I suspect that if teachers were asked to formally assess yet another aspect of children’s lives and performance, they would revolt. They would also probably feel less than competent to make these kind of judgements, though with a little reflection, they would certainly be skilled enough to help children to reflect on their social and emotional competence and create the kind of climate which would engender discussion about difference and maybe more appropriate ways of handling themselves and/or situations’ (teacher/researcher). From a practical perspective, the national programme practitioner who had experience of overseeing 30 schemes said ‘Very varied! Generally our checklist is accepted because it is not particularly time consuming and the scheme is not an intrusive intervention for either school or children – in fact it works in a remarkably non-stigmatising way. The likelihood is that a more intense assessment session would provoke more intense views’. 4.9 The importance of the child’s voice – self-assessment Most respondents described the child’s voice as ‘vital’, ‘absolutely essential’, ‘critical and central’ or words to that effect. Respondents felt that without listening to the child, assessment would be unlikely to achieve its aims; ‘without honouring the student voice it will fail or be damaging’ (researcher). Other points raised in this context included pointing out that ‘listening to the child’s voice may become tokenistic and that it is important to listen and hear. Children show feelings and anxieties through actions and not just words’ (principal educational psychologist). As one researcher stated; ‘Emotional literacy is closely tied with self-concept, and it is difficult to see how judgements can be made by an “other” in these areas. The child’s view is paramount’. Including the child’s voice has practical implications, but respondents considered that even quite young children could make a valuable contribution to their own assessment. Some respondents suggested similar solutions such as the need for appropriate presentation and the ‘use of concrete symbols/photos and reword (change the language) so that it is age appropriate with child’s experience’. The practitioner from the national programme was more practical: ‘Whilst the child's voice should undoubtedly be involved in any assessment, effort needs to be given to the most appropriate method for doing this. Our schemes will generally use simple













33

pictograms (drawings of cartoon-like characters depicting emotions) at the outset of the club sessions and repeat the exercise, together with verbal, written, pictorial representations at the end. Several schemes are considering introducing sociograms (similar to pictograms, but depict a social situation) as an assessment method and we will be particularly keen to assess how this has impacted on the scheme’. In her review paper, John (2001) advocates including the child’s voice in assessment and suggests that children as young as six years can contribute. Harter (1993), in developing her self-perception profile for children, stated that from the age of about eight years onwards, children were quite capable of describing their self-perceptions across six different domains including global self-worth and social competence. Even younger children have been able to provide information predictive of social competence. Emotional perceptiveness scores derived from self-report by pre-school children, predict adaptive social behaviours and academic competence four years later after controlling for components of intelligence and personality, with high scores predicting positive social behaviours and low ones predicting problem behaviours (Izard et al., 2001). 4.10 Other issues raised Three of the respondents (PSHE advisor, educational psychologist and a researcher/advisor) raised the issue of assessing the school. The researcher/advisor highlighted her experience of ‘monitoring the effects on [inappropriate] behaviour and attitudes in the school, which demonstrated the transference of emotional literacy skills to real situations.’ For example in one school the number of “warnings” given to children was monitored over time and shown to fall significantly. In the same school the ability of the pupils to take responsibility in decision making was observed to increase and teachers reported much lower stress levels. Children also showed an increased ability to resolve conflicts without resort to adults and there was a fall in the number of “outbursts” by children. In the view of this researcher, ‘assessment on this scale is just as informative of the effects of intervention as the assessment of individual children’. She was highlighting the importance of assessment of the whole school and the demonstration of generalised improvements in behaviour and attitudes. All children benefit from these changes. This approach lessens the need for all staff to be highly emotionally competent and lessens any stigma attached to focusing on individual children. 4.11 Purpose and benefits of assessment – the ‘so what?’ issue. What are

assessments used for? This issue provoked some of the longest responses of all. Respondents said these sort of comments to us:- ‘The whole issue of assessment … is a thorny one’ (advisor/researcher). ‘The use of the phrase emotional competence implies that you can also be emotionally incompetent and I have a problem with that’ (researcher). ‘They should not be just performance indicators for the establishment’ (teacher/researcher)’. Some respondents had serious concerns about how the information would be used (practitioners). Others questioned the existence of emotional competency as a separate entity (educational psychologists). These responses suggest that assessment of emotional competence will need to be introduced with caution and that sensitivities need to be taken into account.







Deleted: ’

Deleted: ’

34

One area of sensitivity is that assessment will be judgemental. An inclusion co-ordinator said ‘I think the reason for assessment is to inform the way that we promote it [emotional competence]. Most importantly to find out if our approach to working with young people in this area is effective and secondly, to identify and support those who find it more difficult. Once assessment is deemed to be judgemental then this will erode the work that we are doing to promote these skills’. Other respondents expressed concern about the accuracy of assessment tools. ‘We should be humble about what any assessment device actually tell us – there is not any absolute truth but some indications of how a child is developing’ (PSHE advisor). Another respondent reported ‘being emotionally competent is an aspiration for all children and not just the problem ones’. Many respondents from different backgrounds considered the major benefit of assessment to be helping the child; ‘Our team feels strongly that assessment instruments are only beneficial if the information is used to guide the development and implementation of plans that benefit children’ (US researcher). However, not by judging the child; ‘They should only be used to assess how effective we are at promoting these skills with a view to enhancing the skills/competencies not to judge how emotionally competent/intelligent a child might be’ (inclusion co-ordinator). The concern raised by the whole issue of emotional competence assessment is illustrated by the following; ‘If assessment is used for target setting and [setting] performance criteria, then I will despair. The problem with incorporating a greater emphasis on the personal, emotional and social aspects of growth and development into the current educational context is that the former is philosophically at odds with the latter. Teachers and children will become increasingly confused about their roles in the system and the likely outcomes are anger or apathy’ (advisor/researcher). A further issue for teachers/ practitioners was that they perceive little help to be available to children who they have found following assessment to be in need of help. Consequently, they questioned the purpose of completing the checklist. ‘What is the point of assessing children if there is no extra help, time or resources for them’ (PSHE co-ordinator). Even when respondents identified a clear purpose for assessment, they still warned of adverse consequences; ‘If assessment cannot be matched by appropriate service provision, it will undoubtedly result in cynicism towards the process’ (national programme practitioner). Examples from the literature showed that measures have been used for many purposes, but two of these predominate. The first is to identify children in need of special support in the development of these competencies (screening). The second is to measure changes in mean levels of, for example, social competence in school populations as a consequence of intervention. The use of instruments in formative assessment to profile a child in order to find ways of supporting their development was not covered to any great extent. The concerns of many of the respondents we spoke to was summarised very eloquently by the following; ‘Assuming the term “emotional competency” could be satisfactorily defined (we could not agree a definition) it would have to be recognised that our personal levels of emotional competency vary from hour to hour, day to day











35

and year to year. The very use of the term “emotional competency” raises problems as it implies that this exists as an entity in itself. As a group of educational psychologists we contend that there does not exist a “gold standard of emotional or social competency” against which everyone – adult, young person or child – should/could be judged. If such competencies exist and if they are measurable, they must be acknowledged to be context-based. When assessing any aspect of a person, and most importantly when assessment will look at that person’s core values, it is ethically indefensible not to consider the person holistically i.e. in relation to all other attributes of themselves, taking account of the norms of their social group, their family, their peer and their reference groups’. ‘If it is agreed by others that emotional competency exists and is a measurable entity then the “so what” question must be asked of the whole of society. Do we want a uniform level of competency? If anyone falls below the acceptable level do they lose their job, their rights as an individual or as a parent, their access to education? Who would be doing the judging and for what purpose? This is so value-laden that as working psychologists we would wish to play no part in the measuring or the judging unless it arose from the child’s own perceived need’. As a balance to the general negative concerns, we include an actual example of assessment having a positive impact in a school from an advisor/researcher. ‘The work on emotional literacy formed part of a whole school improvement action plan. We were trying to effect change on behaviour and attitudes in the school, and saw developing emotional competence as the key both to this and to developing motivation and independence, resulting in higher attainment. The assessment was therefore used to demonstrate progress towards these targets on a whole school basis, and was tied in with other factors such as behaviour records. This was very productive. The purpose in this case was not to focus on individuals. The tool could have been used to highlight areas of strength and weakness in individuals and inform teacher planning. There were certainly individuals who stood out as having difficulties in some areas, and had anonymity not been promised, this would have useful information for the teachers concerned’. It is clear from our discussions with people working in early years and school settings that there is a lot of interest in the development of emotional and social competence in schools/settings and recognition of its importance. Most respondents recognised that assessment could be valuable, but at the same time there were serious concerns about how assessment might be used. Although these concerns may have little grounding in reality they would be likely to get in the way of attempts to introduce assessment and would need to be addressed with teachers and practitioners beforehand.





36

5. Instruments and Assessment Frameworks This section describes the 58 instruments we identified, classifying them by their type and purpose. These instruments represent an interest in this area of work from around the world. The first part details the 25 instruments identified primarily by electronic searching of the academic literature. Here instruments were well validated with good reliability. Some focused specifically on identifying children with social incompetence and behavioural disorders and tended to focus on undesirable behaviours from an adult perspective. Over half of the instruments focused on problem behaviour, only a small number including a component of emotional state or development. The second and third parts cover the 33 instruments identified through contact with those researching and working in the field, the second part dealing with the 23 which were complete, but which were not identified in our electronic searches of the literature and the third part with the 10 instruments in development. We found a continuum from those covering only social competence to those covering only emotional competence, with most covering only social competence and the next largest group covering some aspects of both. These instruments differed from those described in part one in that all of them included at least some statements or questions about social or emotional competencies as opposed to ‘incompetencies’. These instruments had been subject to varying degrees of validation and reliability testing. Others represent the efforts of schools and settings to assess emotional competence or of those developing instruments with a specific focus such as reintegration into mainstream schooling. Development of these instruments reflects the change in attitudes detailed in the section 2.2 - the historical context of assessment. A shift is observable from instruments in which the contents focus on observed socially undesirable or non conformist behaviour to those focusing on the child’s strengths and the emotions underpinning behaviour. The development of some later instruments, but not the earlier ones has been iterative, and has often included the views of children. Most of the early instruments have been psychometrically tested, but this was not always the case for the later instruments where such testing was deemed by some to be inappropriate. The third category covers instruments still in development which were more likely to focus on assessing emotional competence. Several adopted innovative methods of testing. Their state of development was such that we were not able to see all of these instruments and few have yet been subjected to all the necessary validity checks.. The descriptions of these instruments were true at the time of writing, but these may change during the course of their future development. We identified a total of 58 instruments that aim to measure some aspect of emotional and/or social competence to some degree. Each instrument is listed in the tables of Appendix II. We have divided the instruments into three groups depending on whether we identified them from publications in the academic literature or from contact with researchers and practitioners active in the field. The latter group has been subdivided into instruments whose development is completed and those that are still in development. The instruments have been developed in a range of countries including the USA, the UK, Canada, South Africa and Belgium. In Appendix II, the contents and method of development of each instrument are described together with evidence relating to their reliability and validity. In a second table, we describe for each instrument the age range for which it is appropriate, who completes

37

it and in what setting, and in the third table we comment on the development of the instrument and the purposes for which we consider that it is suitable, together with any relevant comments from authors. Instruments are identified by their abbreviated names because this is the way they are most commonly described in the literature. The full title of each instrument is listed on page 7 in alphabetical order. 5.1 Instruments identified in the academic literature We identified 25 published instruments and assessment frameworks covering aspects of emotional and/or social competence. The first group include well-validated instruments with good reliability, developed with the aim of identifying children with problem behaviour. Their contents tend to focus on behaviours that adults regard as undesirable. The second group covers instruments measuring social competence. They also include some items covering aspects of emotional development. In several of these, the emotional component reflects emotional states (smiles a lot, seems unhappy) rather than emotional competence. The third group includes instruments which do not fit neatly into either of these groups. They have been included to illustrate the different methods and approaches used to assess social and emotional competence. We identified one instrument which aims to measure aspects of the social competence of schools/settings, as opposed to children. Together, all these instruments serve to illustrate historical approaches to assessment and the variety of methods which can be used to make assessments. 5.1 a Measures focusing on problem behaviours Most of the instruments we identified in this category were developed in the early 1990s. They requested information from teachers about children’s behaviours, including how they interact with others. Most of the items in these measures focus on a range of negative behaviours such as fighting, damaging property or self-exposure rather than children’s strengths. They also cover items such as over-reaction and withdrawal, which are arguably components of social competence. Some of these instruments are related. For example, a series of instruments, the Child Behaviour CheckLists (CBCL), have been developed by Achenbach (Achenbach, 1991; Achenbach & Rescorta, 2001). Some of these instruments were developed in the 1980s and therefore the development papers were not picked up in the searches. The CBCL (Costenbader & Keller, 1990) and the CBCL-TRF (Teacher Report Form) (Harris et al., 1993) are parent and teacher versions of the same instruments covering anxiety, withdrawal, depression, unpopularity, self-harm, obsessive/ compulsion, inattention, over-reaction and aggression. In addition to the original CBCLs, there are also two instruments for pre-school children; the Child Behaviour CheckList for 1.5 to 5 years and the Caregiver-Teacher Report Forms for ages 1.5 to 5 years (Achenbach & Rescorta, 2000). The HCSBS (Lund & Merrell, 2001; Merrell, 1993) and the School Social Behaviour Scales (SSBS) (Emerson et al., 1994; Merrell, 1993) were developed by the same author for parents and teachers (respectively). The parental instrument (HCSBS) has two scales, one focusing on conformity, the other on problem behaviours. The teachers version has two scales with three defined sub-scales in each. The first scale includes interpersonal skills, covering items concerned with judgements about how other children respond to the child in the school environment; self-management

38

focusing on conformity; and academic achievement. The second scale covers:- hostile/irritable, anti-social aggressive and disruptive/demanding behaviours. These instruments have been revised recently (HCSBS, Merrell and Caldarella, 2003; SSBS, Merrell, 2003). The Conners’ Teacher Rating Scale (CTRS-28) (Fantuzzo et al., 2001) is another teacher rating scale covering three constructs, social behaviours (e.g. temper outbursts), hyperactivity and passivity. The last of this type of instrument, the Social Skills Rating Scale (SSRS) (Gresham and Elliot, 1990;Elliot et al., 1993) was designed to assess children with significant behavioural problems. These instruments are typical of early approaches to identifying social competence focusing on behaviours that adults regard as undesirable. They capture behavioural outcomes expressed during social interactions in the school setting. As their main purpose is to allow adults to identify problem children, normal or pro-social behaviours are not covered and children who do not have significant problems are likely to get low scores. These instruments can have significant ‘floor’ effects, meaning that most children achieve the optimum score. They do not therefore discriminate children with mild problems. 5.1 b Measures covering some aspect of emotional development Seven of the instruments covered some aspects of emotional development or state (e.g. levels of sensitivity, miserableness or aggressiveness), but not all of these were directly relevant to emotional competence. Four were designed for younger children and in three of these (Infant and Toddler Social and Emotional Assessment, ITSEA; Penn Interactive Peer Play Scale, PIPPS; and Preschool Behaviour Checklist, PBCL), social competences and behaviours dominate the emotional components. The ITSEA (Briggs-Gowan & Carter, 1998) although designed for very young children (1-2 years) was developed with 2-3 year-olds and can be used with 3 year olds. This instrument aims to assess socio-emotional competences and complements other diagnostic tools. One of the five scales, the ‘competencies scale’, includes the emotionally relevant sub-scales empathy, emotional positivity and emotional awareness. Pro-social behaviour and peer relations are also covered. The ‘competencies scale’ also includes attention skills and compliance so scores do not measure emotional and social competence alone. This scale provides an example of positive items e.g. ‘smiles a lot’, ‘tries to do as you say’, but the other four scales in this instrument are worded negatively and focus on problem behaviours. These include in the externalising scale ‘hides misbehaviour’, in the internalising scale ‘is very clingy’, in the dysregulation scale ‘refuses to eat’ and in the maladaptive scale ‘ swears’. PIPPS (Fantuzzo et al., 1995) and PBCL (St James-Roberts et al., 1994) both include a small number of items recording emotional states (smiles, seems unhappy and miserable, fearful) and also desirable social actions (helps others). The other items in both scales are concerned with a lack of social competence and in the case of the PBCL, developmental/behavioural items (wets, soils, unclear speech). FOCAL (Mitchell-Copeland et al., 1997) is a research tool that records event-based emotional interchanges in which children’s emotions are noted (happy, sad, angry, afraid, hurt, tender etc.) as well as their responses to others’ emotions (positive or negative reinforcement, helping, comforting, affect match – positive, opposite or

39

negative, looking, ignoring etc). Pre-school children are observed for five minutes on 12 occasions in free play in the classroom and the scores derived from the observations form the basis of the assessment. Three instruments aimed to assess social and emotional development in older children. One of these, the Interpersonal Competence Scale (ICS) (Cairns et al., 1995) covers academic competences (e.g. good at spelling, math) as well as social competencies (e.g. popular with boys/girls) and emotional states (e.g. smiles a lot, argues, cries). The SDQ, a UK developed instrument (Goodman, 1997), includes positive items such as ‘considerate of other people’s feelings, being helpful, unhappy and fearful’. The remaining items predominantly reflect problem behaviours. This allows raters to assess children from a positive and negative perspective. The Separation and Anxiety Test (SAT) (Duffy & Fell, 1999; Wright, Binney & Smith, 1995) directly assesses the emotional awareness and insight of children aged 8-12 years. Children are presented with vignettes and photographs and asked to say how they would feel if the stories were happening to them and how they think the people in the photographs feel. This instrument was designed to assess attachment, and responses are therefore scored on the constructs of attachment, self-reliance and avoidance. The Behavioral and Emotional Rating Scale (BERS) (Epstein et al., 2002) is an instrument with five subscales three of which are relevant to the measurement of emotional and social competence. The ‘affective strength’ subscale is particularly relevant, including items such as ‘identifies own feelings’ and ‘ability to give and receive affection’. The ‘interpersonal and intrapersonal strengths’ subscales are also relevant: items include ‘shows concerns for the feelings of others’ and ‘manages anger effectively’. The subscales for school functioning and family involvement are less relevant. Although this instrument is designed to identify children with emotional and behavioural problems who are in need of extra support (screening), all the statements are phrased in a positive way (e.g. accepting of affection rather than rejects affection). It was developed following research undertaken with parents, children and teachers and therefore represents more than the ‘expert view’. It is suitable for children aged 5-18 years. At present it is designed for adults to complete, but a self report version is in development. This instrument was one of a small number identified in the published literature which is relevant for the assessment of emotional competence. It might be useful for monitoring and profiling. Although developed and recommended for screening it would need further testing in this context. The Emotional Instability, Prosocial Behaviour and Aggression Scales (EIPBAS) (Caprara & Pastorelli, 1993) assesses emotional instability, prosocial behaviour and aggression and can be used as a self-report instrument by 7-11 year-olds and by adults. The emotional assessment here is concerned with the child’s capacity to refrain from impulsivity and emotionality. 5.1 c Other instruments We identified several instruments that are potentially interesting in the context of the assessment of emotional and social competence that do not fall neatly into either of the above categories. Two of them, the Bully-Victim scales (Austin and Joseph,

40

1996) and the Diagnostic Interview for Social and Communication Disorders (DISCO) (Wing et al., 2002) were recently developed in the UK and so have a cultural relevance. The first examines perceived bullying and victimisation in mainstream children and is one of the few self-report instruments that we identified. The relevant items have been added on to the Harter Self Perception Profile (Harter, 1993) The second instrument (DISCO) is an extensive interview which aims to establish the diagnosis of autism. These scales were developed with mainstream children as well as those with special needs, and therefore cover the continuum of competence in social interactions. The Social Ability measure (Braza et al., 1993) is a rather different type of instrument, which assesses social competence on the basis of diversity of social contacts and behaviour. This observational instrument was designed to gauge 5 year-olds’ social development for research purposes. The following instruments were included to illustrate the diversity of potential approaches to measurement of these concepts. The Penn State Worry Questionnaire for Children (PSWQ-C) (Chorpita et al., 1997) is designed to elicit anxiety and asks children about worry. This is another self-report measure. Children are asked to assess an internal trait without recourse to behaviours or social situations. For example, they are asked to rate their ‘worries’ (e.g. my worries really bother me) hypothetically and not in relation to specific circumstances. The Social Phobia and Anxiety Inventory for Children (SPAI-C) (Beidel et al., 1995) is another measure of anxiety designed for teachers to complete. It is interesting because it is based on the ability of adults to assess children’s emotional states through observation of behaviour. The Child Behaviour Rating Scale (CBRS) (Broder et al., 2001) like the DISCO was developed with children with and without specific medical conditions (in this case, cranio-facial abnormalities). It examines social interaction. The Dominic-R: A Pictorial Interview (Valla et al., 2000), was designed to detect mental disorders in 6-11 year-olds. The instrument is a series of pictures (of Dominic, who can be male or female) in varying situations. Children are asked to talk about the pictures and their responses enable the diagnosis of simple phobias, separation anxiety, over-anxiousness, depression, attention deficit hyperactivity disorder, and conduct disorders. 5.1 d Instruments assessing social and emotional competence of

schools/settings. The Child Development Project (Solomon et al., 2000) is an instrument that aims to measure the social wellbeing of the class and school. It was developed in the context of a programme to help teachers create a caring community in the classroom and school. It is a student report instrument in which two thirds of the items are phrased positively. It covers the three key attributes of healthy interpersonal relationships:- respect, trust and empathy. Whilst this instrument does not directly touch on emotional competence, the development of a caring classroom or school would require such skills from the teachers and pupils. The items include statements which imply emotional competence e.g. ‘when someone in my class does well, every one in the class feels good’ and ‘people care about each other in this school’.

41

5.2 Completed Instruments identified by contact with researchers and practitioners

We identified 23 instruments, which our searches did not identify in the academic literature, with one exception, through contact with researchers and practitioners working in the field. These instruments differ from those in the previous group, in that none focused solely on problem behaviours and all contain some positively phrased items. Several contain items which measure emotional competence. 5.2 a Measures focusing on primarily social competence One group of instruments aimed to detect children with emotional and behavioural difficulties, but these differ from instruments identified in 5.1a in that they cover both positive and negative aspects of behaviour. Few have items which touch on emotional competence in the sense that it has been defined in this report, but some include aspects of emotional wellbeing and resilience. The Adaptive Social Behaviour Inventory (ASBI) (Hogan et al., 1992) is a teacher rating scale that assesses social competence in 3-5 year-olds on three scales, expression, compliance and disruption, with the aim of identifying emotionally and behaviourally disordered (EBD) children. Clarbour and Rogers considered this scale when developing their more recent Emotional Behaviour Scale (EBS) (Clarbour and Rogers, in press) which assesses adolescent emotional coping strategies and focuses on social anxiety, malevolent aggression and social self-esteem. The latter is aimed primarily at adolescents but can be used with children as young as eight year-olds. The Devereux Early Childhood Assessment Program Instrument (DECA) (LeBuffe & Naglieri, 1998) was developed in the US and has versions for teachers and parents, covering initiative, self-control, attachment and behaviour, and is based on resilience and protective factors. This instrument assesses aspects of emotional competence including expressing feelings appropriately, ability to share, seeking help and showing interest and co-operating with others. (e.g. listen to or respect others and calm himself/herself down when upset), with more than half the items phrased positively. It is appropriate for 2-5 year-olds. It can provide individual profiles as well as class profiles and is potentially useful for screening and monitoring. The Boxall profile (Bennathan & Boxall, 1998) grew out of the Nurture Group movement. It seeks to identify the areas of difficulty in accessing education for children entering school from severely disadvantaged backgrounds and to help teachers plan focused interventions and monitor progress. It is standardised on 3-8 year olds. Both the Boxall profile and the DECA are part of programmes to support children with disordered development – the DECA supports a class-based approach and the Boxall Profile a nurturing approach in a group. The Fast Track programme in the US, an intervention primarily designed to prevent serious and chronic antisocial behaviour, has been associated with the development of a series of instruments (Greenberg et al. website, no date). These include a multifaceted series of measures – child interview, and parent and teacher rating scales for children of varying ages. The emphasis of Fast Track instruments is on social rather than emotional competence and they tend to focus on negative behaviours. 5.2 b Measures which include assessment of the impact of social and

emotional competence on learning A second group of instruments assess social competence, emotional development and behaviour, but in the context of their impact on learning. Those developing these

Formatted: Not Highlight

42

instruments include emotional and social competence in their scales because they are clear that they have an impact on learning. These are all teacher-rating scales. The Early Years Profile (Hereford & Worcester Partnership, 1997) is completed on entry to full time education and is intended to reflect a child’s attitude, attributes and understanding mostly about educational subjects. The Short Term Education and Pupil Support (STEPS) is a teacher rating scale developed by Stockport Borough Council for measuring social skills related to learning which has a different construction. Each item has positive and negative aspects, e.g. attentive and listens vs distracted and inattentive respectively. The scoring is on a 1-6 scale, with positive behaviours scoring 6-4 and higher frequencies of negative behaviours scoring 3-1. The Learning Involvement Scale (LIS-YC) (Laevers, 1994), is designed for use with 3-5 year-olds to assess the extent to which children are emotionally engaged in the process of learning. Assessors have to be trained using videos. This instrument can be used with both individuals and classes, with versions for other age groups including adults. The Emotional and Behavioural Development Scales (EBDS) (Riding et al., 2002) can be used with a larger age range as the descriptors can be applied to 5-16 year olds. This is a brief checklist that covers social/emotional and development behaviours with one third of the instrument about learning outcomes. Another UK instrument, the Pupil Attitude to Self and School (PASS) (Goodall, 2002), assesses attitudes to self and school in relation to learning. This has a similar age range, 8-16 years and covers 50 child self-completion items. This has standardised norms for the UK population and can be used to assess the individual child or the school. One instrument, the Early Development Instrument (EDI) (Offord & Janus, 2001) is a population-based measure for communities, which assesses readiness to learn in kindergarten children. This Canadian instrument has a comprehensive section on social competence and emotional health, which does not directly assess emotional competence, but also includes physical health, cognitive development and communication skills. There is some evidence that the EDI is predictive of academic outcomes several years later. Most instruments do not specify when they should be used, often because it is not appropriate to do so. However, the two exceptions were the EDI, which is recommended for completion by kindergarten teachers in the second half of the school year, and the Early Years Profile that should be used within seven weeks of the child starting school full time (Hereford & Worcester Partnership, 1997). Two of the instruments developed in special schools look specifically at children’s readiness for reintegration into mainstream schooling. These are the Coping in School Scale (CISS) (McSherry, 2001; originally called the Reintegration Readiness Scale in 1996) and the Reintegration Readiness Scale (Doyle, 2002). The first has general sections: – ‘self management of behaviour’ and ‘self and others’ – and short sections for specific behaviours: self-awareness, confidence and organisation, attitude, learning and literacy skills. The instrument can be used to support reintegration of older primary children before they change to secondary school and to monitor problems through the transition period. It contains positively phrased items, is child-centred and was designed for Year 6 children, and occasionally younger, to complete with adult support. It highlights areas that need addressing, with the emphasis on self-management and learning, but has few emotional items. The second is based on the CISS, but is shorter and can be used with 7-11 year-olds.

43

The last instrument is interactive and seeks to identify gaps in emotional development so that reparative work can improve the deficits. It is therefore intended for use only in children who have been identified as having emotional or behavioural difficulties. It was developed as part of the Enable Project (Banks et al., 2001), is computerised and can be completed by the teacher, parent or child (children are asked their opinion about statements if too young to complete the whole instrument). Statements are selected from various topic areas describing predominantly negative behaviours of subjects. The six most relevant statements are chosen to identify strategies to repair missing developmental stages for the child on a personal level, and in the classroom, related to the National Curriculum. This instrument also has the capacity to be used by adults about themselves. The instrument also has positively phrased monitoring statements used to follow children’ s progress. These are currently being adapted for use as a screening tool in schools (Gerlach, personal communication, February 2003) 5.2 c Measures focusing on emotional competence We identified two instruments recently developed for use in English schools and settings which are specifically aimed at assessing emotional competence. One is a teacher report instrument – the Record of Assessment for Emotional Literacy Checklist – which covers three aspects of children’s awareness: handling relationships, managing and knowing own emotions, and recognising emotions in others. This instrument is being used in Mason Moor Primary School, Southampton and was developed by the Southampton Emotional Literacy Interest Group. This group is interested in early years and primary school aged children. The instrument can be used to assess emotional competence both at the individual and class level. The second was developed by New Close Primary School (McCalley and Potter, 2002). This one is designed for children aged 10-11 years and the self-report part includes ‘How I see myself’ where children rate themselves on 11 items and complete 12 written statements (e.g. today I feel …). Teachers also rate children’s emotional intelligence, but the children’s items are interpreted as a mixture of social, emotional and particularly learning outcomes (e.g. capable of counselling other children who have difficulties and shows empathy; can mediate disputes; generally optimistic; positive about learning). Scores from both children and teachers are entered into a matrix. Both of these instruments were developed within the schools and so none of the statements have been validated, tested or standardised. The ‘Taking Care Project’ in Sheffield developed an interview-based assessment which uses scenarios of bullying and domestic conflict to illicit feeling words and help seeking intentions with 5-11 year-olds. Discussion of these issues is addressed through school activities such as circle time and expressive drama. There has been no standardisation or evaluation of which we are aware and so the interpretation of the children’s responses to these may be at an individual teacher level. The EQ-i:-YV(S) (Bar-On & Parker, 2000) is a short questionnaire (both 60 item and 30 item versions are available) in which items were derived from the authors clinical experience, covering various aspects of emotional competence (e.g. it is easy to tell people how I feel), social competence (e.g. I can tell when one of my close friends is unhappy), management of emotions (e.g., I get too upset about things) and problem solving (e.g. I can come up with good answers to hard questions). The EQ-i:-YV is

44

applicable for 7-18 year-olds with a special response sheet for the younger children. There is a version of this instrument for adults, the EQ-i. The early development of this instrument is described in a PhD dissertation which we have yet to obtain, but subsequent development has been well documented in the technical manuals for the EQ-i and EQ-i:YV. These give details of internal and test-retest reliability tests, factor analyses, and construct validity. Population norms are also described. The EQ-I:-YV is suitable for profiling and monitoring and possibly for screening. The Process-oriented monitoring system (POMS) (Laevers et al., no date) was developed to assess children’s wellbeing and involvement in class. The instrument covers developmental and educational areas (e.g. motor, language, understanding). The well-being assessment covers the quality of children’s relationships with others and high levels would depend on good emotional and social competence, but the instrument does not specifically focus on emotional competencies. It has three stages. The first stage screens the whole class for children with socio-emotional or developmental problems. This stage has two options for assessment: Variant A has three levels (low, medium high) on which to assess children and Variant B has five (1-5 for greater differentiation). Teachers can choose to observe the class on either variant depending on their level of competence. The manual contains examples of wellbeing and involvement at each level, together with conclusions, interpretations and suggestions for interventions. The second stage is a closer observation of individual children and analysis of their behaviour, with the intention of identifying the children with low wellbeing scores. Here the child’s wellbeing is assessed in four domains of social activity (peers, teachers, family and the children’s play-, class- and school-world), with more detailed descriptions set into a framework. Again, instructions, examples of interpretations of wellbeing and involvement are included. Stage three is setting out the goals for action where the children are assessed in context, the seriousness of any problem established and the areas which are to be addressed. 5.2 d Other Instruments A variety of other instruments were identified including a worksheet called Cogs (Ball, 2002), with 10 segments for the child to fill in as a form of self-assessment of areas such as schoolwork, social competence and self knowledge. This is an approach to measurement which can be carried out as a class exercise. Most of the items are not relevant to emotional competence. We identified one instrument which was specifically designed to evaluate a school emotional literacy intervention. Mary Layton devised a structured interview covering 13 vignettes with coded responses for use in 5-8 year-olds. The instrument is mostly concerned with social skills but has questions about awareness of others. We were also told about two potentially relevant instruments, which we have not been able to find in the timescale of the project. The Optimistic Child scale (Seligman & Rockwell, 1996) developed in the US, is an assessment for children aged 8-12 years measuring their explanatory style (optimism vs helplessness). The Self-Esteem Indicator (Morris, 2001), a brief checklist for Key Stage 1 teachers, assesses sense of personal power, sense of self and sense of belonging. This indicates where children need help and is supported by a booklet which suggests ways of improving self-esteem in children.

45

5.3 Instruments in development identified by contact with researchers and practitioners We identified 10 potentially relevant instruments which are currently in development. Most of these are instruments which we have been told about, but have not yet seen. We are therefore not in a position to do more that relate what we have been told about their contents and development. In contrast to those in the previous sections, all but one of these instruments are being developed in the context of English schools and settings primarily to assess emotional and social competence. The first is an instrument, which is based on the work of Professor Laevers (Laevers, Personal Communication, 2002) at Leuven University, the author of the Leuven Involvement Scale covered in section 5.2a. He has also developed a resource pack to improve emotional and social competence, called the Box Full of Feelings. This provides materials covering the four basic emotions of anger, fear, sadness and happiness. Professor Laevers is currently overseeing the development of a new instrument, the Social Competence Test (SCoT), that assesses children’s emotional and social competences to accompany the Box Full of Feelings. An assessor rates children’s responses to a set of videoed vignettes. The SCoT measures all aspects of emotional competence, and is being developed for research purposes. We have not been able to view the instrument because it is not yet at that stage of development, but we are told that the instrument assesses the emotional and social responses of children 4-8 years. Trained researchers show children 12 scenes displayed by puppets and two fragments of a ‘Mr Bean’ video individually. The videos are stopped several times following a standardised procedure. Children are asked about the following: role taking (imagining themselves to be the character depicted in the scenarios) with respect to feelings and emotions, perceptions and cognitions, intentions and motives, and their understanding of the observed behaviour, predicting behaviour and acting in a socially competent way. Children are scored on a four-point scale based on categorisation of answers where the main focus is on emotions and their development for the child. The SCoT has been evaluated both in terms of the quantitative data and in terms of qualitative responses from children, but needs further reliability studies. It takes 25 minutes to complete and practitioners/teachers have to be trained to distinguish between different levels of behaviours or responses. The Accounting Early for Life Long Learning, AcE Project (Pascal & Bertram, 2001) is being developed at University College, Worcester. The instrument is a practitioner rating scale detailing the specific observations which practitioners need to make and record, covering indicators of attitudes, dispositions and emotional wellbeing. The indicators cover emotional competence (e.g. express and work with own emotions, empathy, appreciate moral culture); empowerment (e.g. unafraid, confidence to explore, pride, purposeful); connectedness (e.g. attachment to people, sense of belonging); positive self-esteem (e.g. demonstrate confidence, respect own beliefs, resolve conflict, sense of humour, express values). The instrument has been developed in order to help practitioners improve the learning experience of 3-6 year-olds, and enables the identification of children who need extra support. The latest version of this instrument is being piloted in the academic year 2002 to 2003. We have not seen the instrument and so cannot be sure of the emotional competence content. As described to us it is similar in approach to the Learning Involvement

46

Scale where teachers are trained with videos to standardise the assessment of children. It appears to have a learning focus. A further instrument, the Feelings and Empathy Questionnaire (Browning, 2002) being developed to improve emotional literacy in a primary school, is being described as part of a Master’s degree nearing completion. The author told us that the instrument was developed with a small number (<20) of 8-11 year-old children and has three sections. The first has six shapes where children are invited to write about ‘what other people think of me’ including friends, siblings, parents, teacher, head teacher and others. The second ‘developing an emotional vocabulary’, asks children to write down as many feeling words as they can and then to explain them. They are then asked to listen to a short story and write down how they would feel and what would they have done if they were the two characters in the story, which is designed to assess empathy. Scoring is based on the number of emotional descriptors in section one and numbers of words listed in section two. In the last section, empathy or lack of it is scored on a three-point scale. We have been sent two further instruments whose development is nearing completion, associated with Bristol University. The first, Talkit (Tew), is being developed as part of a PhD and includes items relating to emotional and social competence. It is based on statements children have made about themselves and is primarily a self report instrument. It has five subscales: controlling emotions, inter-personal empathy, keeping motivated, confidence in different situations and a small scale for integrity. There are no explicit items on emotional awareness but the first two subscales include relevant items, (I say what I feel; I notice when other people are upset; I can say I am sorry when I am wrong; I try to make people feel wanted). This instrument was developed with 11 year-olds. It will be computerised and compatible with SIMS (school computer software). It has the potential for profiling children because self, peers, teachers and parents can complete it. The author is interested in developing versions for younger and older children. The second instrument in this group, the Effective Lifelong Learning Inventory (ELLI) Effective Learning Profile (Broadfoot et al., 2002), assesses attitudes and approaches to learning (I learn a lot by watching and listening to other people). There are no explicit questions on emotional or social competence, but the authors consider that these competencies underpin, and are inseparable from, attitudes and approaches to learning and that the instrument would therefore identify children who were failing in this respect. It has been tested with 8-11 year-olds and two older age groups, with a version for younger children in development (JELLI). We were told about three approaches to assessing the emotional and social competence of schools/settings rather than children. The Emotional Literacy Audit, ELA (of organisational culture) is being developed by Antidote, the UK Campaign for Emotional Literacy, as a tool to foster emotional literacy in schools. The items in this instrument are based on interviews and focus groups with children and school staff. The Graduate School of Education in Bristol is collaborating with the development work, which is extensive. The ‘What I think about my school’ (Roche, 2002) scale is a pictorial questionnaire for 7-10 year-olds and was developed as part of an action research project with four schools in Nottingham. It covers aspects of social competence and wellbeing at school level. This instrument is being developed as part of a Healthy Schools initiative, and is open to development. No reliability or

47

validity assessments have been undertaken. An educational psychologist (Priest, Rotherham Metropolitan Borough Council) told us that he has developed an instrument to enable teachers to reflect on their own emotional competence. This is based on Goleman’s formulation (Goleman, 1996) of emotional intelligence (self-awareness, self-regulation, motivation, empathy and social skills) and was designed to promote discussion. We were also told about two further instruments (Sefton Council instrument and a Pre-School Transfer Form in Somerset, Ellis and Jennings, 2002) which are currently being developed to assess emotional competence as part of projects which aim to improve behaviour and/or emotional and social wellbeing. We have insufficient detail about these instruments at present to critically appraise them but we list them here for completion. The development of these assessments is a marker of the level of interest in emotional competence assessment in the UK at present.

48

6. Discussion and conclusions In this section we discuss issues associated with emotional competence assessment in the light of the comments from the interview study and the instruments we identified. We have found a lot of activity and interest in such assessment. Fifty eight instruments were identified in different stages of development. We identified several instruments which cover many aspects of emotional competence which, between them, could fulfil all the three purposes of early identification, profiling and monitoring, both in individual children and in schools and settings. This area of educational activity is new, very active and in a state of flux. The practitioners, teachers and researchers we spoke to were very interested in developing methods of assessing emotional competence, but there was not universal agreement about what the concept entailed. Those we spoke to also expressed concerns particularly about lack of time for assessment, lack of facilities to help children with problems, a mistrust of checklist type instruments and concerns about the uses to which assessment might be put. During the course of this project we have identified a significant level of interest in the concept of emotional competence. Some of those we spoke to, however, had differing ideas about what this concept, and the related concept of social competence, entailed. Some saw lack of these competences as synonymous with lack of emotional wellbeing, others with problem behaviour. Some were interested in the concept primarily because of its relevance to lifelong learning and school achievement, others because they felt it was important in its own right. An issue for further debate is the extent to which emotional and social competence encompass the attribute of autonomy. Many working in the field in the past have implicitly assumed that social competence, particularly among children, implies an element of social compliance. Others have suggested that autonomy is important for personal development and lifelong learning and that it makes a valuable contribution to social and emotional competence. A considerable number of people responded to our communications about the assessment of emotional competence, telling us that they were working on projects which aim to support the development of these competencies in schools and settings. On further investigation it turned out that some of these people were working to promote emotional wellbeing or reduce problem behaviour. There is therefore still some room for clarification about what these concepts entail. In looking for instruments which could assess or measure emotional competence, we found it difficult to separate these instruments from those that measured social competence because the two concepts are so closely related. However, most early instruments concentrated on social competence, and it is only recently that instruments covering emotional competence, as defined in this report, have started to appear. Some of the instruments which covered emotional as well as social issues focused on emotional well-being rather than emotional competence. We documented a considerable interest in assessing and measuring these competences in the published literature and among researchers working in the UK, Europe and the US. We also made contact with a number of teachers who were developing their own instruments for use in their schools or early years settings. Teachers are unlikely to have the resources at their disposal to be able to evaluate the reliability and validity

49

of instruments they develop, so these are not likely to ‘score’ highly in a review such as this. It may, however, be worth examining the contents of such instruments again to achieve a perspective on what teachers think is important. It is unlikely that we have managed to identify all those who could have made a contribution to this report, and we have reviewed only the literature that we managed to identify and obtain in the relatively short time span of the project, but it is clear from those with whom we did make contact that assessment and measurement of emotional competence is practical and achievable. Instruments which are currently available or in development, could in the future make assessment and measurement for the purposes of early identification, profiling and monitoring a real possibility. Our review of instruments demonstrated an evolving understanding of social and emotional development with contributions being made from a variety of different disciplines. Early instruments were designed by experts and focused on problem behaviours. By contrast, some later instruments were developed on the basis of research with parents and children, and included positive social competencies. Newer instruments enable reporting by multiple observers as well as children themselves, in recognition that emotional and social competence and behaviour can be context specific and that children’s perceptions are important. The methods used for assessment have also evolved, early instruments relying on observer responses to behavioural items in a checklist. Newer instruments, both those we identified in the literature and those we identified by contacting researchers, include some based on observation of children’s emotional and social responses to photographs, drawings or video vignettes. Many of the more recent instruments are now available in computer format. One issue for further consideration is the extent to which children or schools/settings should be the focus of emotional competence assessment. The environment is key in supporting the development of these competencies in schools or settings, and therefore it is possible to argue that the school/setting is the appropriate focus. As one researcher put it we need to ‘move away from assessing the pathology in the child to [assessing] the pathology in the system’. The feeling of being judged and found wanting is a very real deterrent to professional or personal development, and such an approach might prove counterproductive to initiatives to promote emotional and social competence. Almost all those we spoke to believed that early identification of children whose emotional competence was poorly developed was important. Early identification is an approach which is best suited for well-defined conditions (those where there is a clear demarcation between mainstream and special needs) for which there are effective, available interventions. From a theoretical point of view, therefore, emotional competence is not an ideal candidate for early identification, but with careful planning and thought it can be made to work. It is very important, however, if early identification were to be carried out in all settings, that there would be adequate facilities to support the children who are identified as in need. This might mean offering class teachers additional support in managing the children in need in their class. Early identification would also need to be handled with care since it carries the risk of negative labelling of children. Completing emotional and social competence assessments for all children would be less likely to be stigmatising Some of the

50

completed instruments assessing emotional as well as social have been developed with a screening function in mind and are recommended for this purpose, but more work needs to be done to demonstrate their usefulness in this context. In an ideal world all practitioners/teachers would have the time and skills to spot children with poor emotional and social competence without recourse to instruments, and the time and ability to offer them the special support they need, both in the classroom and elsewhere in the school/setting, in a non-stigmatising way, Profiling instruments can help practitioners/teachers identify children with problems and also decide on the areas of development which need supporting. It is useful to note that teachers expressed a distrust of the ‘checklist approach’ to assessment in their experience checklist results were often not followed up. There was a very strong feeling that individual children should be allowed to be the best that they can be, not moulded into a socially compliant being who scores well on a check list. However, most instruments we identified were checklists. They may be presented differently e.g. computerised, but the scoring is still a list against which the respondent is judged. This is inevitable as part of the standardisation process. The key issue is how the statements against which the children are judged are derived. Development which includes participants makes for better instruments, and in these newer instruments children are assessed against statements relevant to them. Self report instruments although presented as a checklist may be more acceptable. Instruments which are primarily based on positive statements are also likely to be more acceptable. The disadvantage of focusing on negative items is that raters concentrate on the child’s inappropriate or problem behaviours and so assess how ‘bad’ the child is. The child may become aware of teachers or practitioners attentiveness to this type of behaviour. By contrast, when items are phrased positively, raters are encouraged to focus on the child’s strengths. In conclusion we found a lot of activity and interest in the assessment of emotional and social competence and concluded that assessment is likely to prove feasible in future. Most of the instruments included in the review focused on social competence, but we identified several which assess aspects of emotional competence and are potentially suitable for use in the three contexts screening, profiling and monitoring. The instruments most relevant to emotional competence assessment were applicable to school settings rather than early years. Practitioners had strong views about how assessment might be used. The value of school assessments as part of the development of whole school approaches were seen as important by some respondents but were not widely understood by teachers and practitioners.

51

References Achenbach T.M., and Edelbroch C.S. (1991) Manual for the child behaviour checklist:4-18 profile Burlington VT, University of Vermont Dept of Psychiatry. Achenbach, T.M. & Resorta, L.A. (2001). Manual for ASEBA School-Age Forms and Profiles. Burlington VT, University of Vermont, Research Center for Children, Youth and Families. Achenbach, T.M., & Rescorta, L.A. (2000). Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families. Austin, S. & Joseph, S. (1996). Assessment of bully/victim problems in 8 to 11 year-olds. British Journal of Educational Psychology, Vol 66, 447-456. Ball. (2002) Personal Communication Banks, T., Bird, J., Gerlach, L., Henderson, M. & Lovelock, R. (2001). ENABLE Emotional Needs, Achieving, Behaving and Learning in Education. The Modbury Group, Modbury, Devon, UK. Bar-On, R. & Parker, J.D.A. (2000). Bar-On Emotional Quotient Inventory: Youth Version. Technical Manual. Multi-Health Systems Inc, New York. Beidel, D.C., Turner, S.M., & Morris, T.L. (1995). A new inventory to assess childhood social anxiety and phobia: The Social Phobia and Anxiety Inventory for Children. Psychological Assessment , Vol 7, 73-79. Bennathan, M. & Boxall, M. (1998). The Boxall Profile. A guide to effective intervention in the education of pupils with emotional and behavioural difficulties. Association of Workers for Children with Emotional and Behavioural Difficulties, London. Bibou-Nakou, I., Kiosseoglou, G., & Stogiannidou, A. (2001). Strengths and difficulties of school-aged children in the family and school context. Psychology: The Journal of the Hellenic Psychological Society, Vol 8, 506-525. Bierman, K.L. & Welsh, J.A. (2000). Assessing social dysfunction: the contributions of laboratory and performance-based measures. Journal of Clinical Child Psychology, Vol 29, 526-539. Braza, P., Braza, F., Carreras, M.R., & Munoz, J.M. (1993). Measuring the social ability of preschool children. Social Behavior and Personality, Vol 21, 145-157. Briggs-Gowan, M.J. & Carter, A.S. (1998). Preliminary acceptability and psychometrics of the Infant-Toddler Social and Emotional Assessment (ITSEA): A new adult-report questionnaire. Infant Mental Health Journal, Vol 19, 422-445. Broadfoot, P., Claxton, G. & Deakin Crick, R. (2002). Learnacy or Lunacy? Assessing the 21st Century Curriculum, University of Bristol.

52

Broder, H.L., Smith, F.B. & Strauss, R.P. (2001). Developing a behavior rating scale for comparing teachers' ratings of children with and without craniofacial anomalies. Cleft Palate and Craniofacial Journal, Vol 38, 560-565. Browning, R. (2002). Personal Communication, August, 2002. Cairns, R.B., Leung, M.C., Gest, S.D. & Cairns, B.D. (1995). A brief method for assessing social development: structure, reliability, stability, and developmental validity of the Interpersonal Competence Scale. Behavior Research and Therapy, Vol 33, 725-736. Caprara, G.V. & Pastorelli, C. (1993). Early emotional instability, prosocial behaviour, and aggression: some methodological aspects. European Journal of Personality, Vol 7, 19-36. Carter, A.S., Little, C., Briggs-Gowan, M.J. & Kogan, N. (1999). The infant-toddler social and emotional assessment (ITSEA): Comparing parent ratings to laboratory observations of task mastery, emotion regulation, coping behaviors, and attachment status. Infant Mental Health Journal, Vol 20(4), 375-392. Chorpita, B.F., Tracey, S.A., Brown, T.A., Collica, T.J., & Barlow, D.H. (1997). Assessment of worry in children and adolescents: An Adaptation of the Penn State Worry Questionnaire. Behaviour Research and Therapy, Vol 35, 569-581. Clarbour, J., & Roger, D. (in press). The construction and validation of a new scale for measuring emotional response style in adolescents. Journal of Child Psychology and Psychiatry. Costenbader, V.K. & Keller, H.R. (1990). Behavioral ratings of emotionally handicapped, learning disabled, and nonreferred children: Scale and source consistency. Journal of Psychoeducational Assessment, Vol 8, 485-496. Demaray, M.K., Ruffalo, S.L., Carlson, J., Busse, R.T. et al. (1995). Social skills assessment: A comparative evaluation of six published rating scales. School Psychology Review, Vol 24, 648-671. Doyle, R. (2002). Personal Communication, August 2002. Duffy, B. & Fell, M. (1999). Patterns of attachment: Further use of the Separation Anxiety Test. Irish Journal of Psychology, Vol 20, 159-171. Elias, M., Zins, J., Weissberg, R., Frey, K., Greenberg, M., Haynes, N., Kessler, R., Schwab-Stone, M., and Shriver, T. (1997). Promoting Social and Emotional Learning. Alexandria, Virginia: ASCD. Elliot, S.N., Sheridan, S. & Gresaham, F.M. (1993). Behaviour rating scales: Issues of use and development. School Psychology Review, Vol 22, 313-321. Ellis, S. & Jennings S, (2002). Personal Communication, August, 2002.


53

Emerson, E.N., Crowley, S.L., & Merrell, K.W. (1994). Convergent validity of the School Social-Behavior Scales with the Child Behavior Check List and Teachers Report Form. Journal of Psychoeducational Assessment, Vol 12, 372-380. Epstein, M.H. & Sharma, J. (1997). Behavioral and Emotional Rating Scale (BERS): A strength-based approach to assessment. Austin, Texas PRO-ED. Epstein, M.H. (1999). The development and validation of a scale to assess the emotional and behavioural strengths of children and adolescents. Remedial and Special Education, Vol 20(5), 253-262. Epstein, M.H., Ryser, G. & Pearson, N. (2002). Standardization of the behavioral and emotional rating scale: Factor structure, reliability, and criterion validity. Journal of Behavioral Health Services and Research, Vol 29, 208-216. Fantuzzo, J., Grim, S., Mordell, M., McDermott, P., Miller, L. & Coolahan, K. (2001). A multivariate analysis of the revised Conners' Teacher Rating Scale with low-income, urban preschool children. Journal of Abnormal Child Psychology, Vol 29, 141-152. Fantuzzo, J., Suttonsmith, B., Coolahan, K.C., Manz, P.H., Canning, S. & Debnam, D. (1995). Assessment of preschool play interaction behaviors in young low-income children – Penn Interactive Peer Play Scale. Early Childhood Research Quarterly, Vol 10, 105-120. Fombonne, E., Simmons, H., Ford, T., Meltzer, H. & Goodman, R. (2001). Prevalence of pervasive developmental disorders in the British nationwide survey of child mental health. Journal of the American Academy of Child and Adolescent Psychiatry, 40(7): 820-827. Gerlach, (2003). Personal Communication, February 2003) Goleman, D. (1996) Emotional Intelligence, Bloomsbury, London. Goodall, S. (2002). Personal Communication, October 2002. Goodman, R. (1994). A modified version of the Rutter parent questionnaire including extra items on children's strengths: a research note. Journal of Child Psychology and Psychiatry, Vol 35, 1483-1494. Goodman, R. (1997). The Strengths and Difficulties Questionnaire: a research note. Journal of Child Psychology and Psychiatry, Vol 38, 581-586. Greenberg et al. Fast Track. http://www.fasttrackproject.org Gresham, F.M. & Elliott, S.N. (1990). “Social Skills Rating System” Manual. Circle Pines, MN: American Guidance System. Harris, J., Tyre, C. & Wilkinson, C. (1993). Using the Child Behaviour Checklist in ordinary primary schools. British Journal of Educational Psychology, Vol 63, 245-260.


http://www.fasttrackproject.org/�

54

Harter, S. (1993). Causes and consequences of low self-esteem in children and adolescents. In: R.F. Baumeister (Ed.) The Puzzle of Low Self-Regard (pp. 87-116), New York, Plenum. Hereford & Worcester Partnership (1997). Early Years Profile. Worcester: Worcester County Council, Education Services Hogan, A.E., Scott, K.G. & Baven, C.R. (1992). EPPE Project. Adaptive Social Behaviour Inventory Child Questionnaire. Journal of Psycho-Educational Assessments, Vol 10(3), 230-239. Izard, C., Fine, S., Schultz, D., Mostow, A., Ackerman, B. & Youngstrom, E. (2001). Emotion Knowledge as a Predictor of Social Behavior and Academic Competence in Children at Risk. Psychological Science, Vol 12, 18-23 John, K. (2001). Measuring children's social functioning. Child and Adolescent Mental Health, Vol 6, 181-188. Laevers, F.R. (1994). Leuven Involvement Scale for Young Children LIS-YC. Centre for Experiential Education, Leuven. Laevers, F., Vandenbusshe, E., Kog, M. & Depondt, L. (no date). A process-oriented child monitoring system for young children. Centre for Experiential Education, Leuven. Laevers, F. (2002). Personal Communication, October 2002, January 2003. Layton, M. (1996) An evaluation of the effectiveness of the ‘School and Family Links Programme’. Part of the ‘Family Nurturing Network’ Registered Charity Number 1034907. Master of Arts in Education Dissertation, Oxford Brookes University Le Buffe, P.A. & Naglieri, J.A. (1998). Devereux Early Childhood Assessment Programme, DECA. Kaplan Companies. Lund, J. & Merrell, K.W. (2001). Social and antisocial behavior of children with learning and behavioral disorders: Construct validity of the Home and Community Social Behavior Scales. Journal of Psychoeducational Assessment, Vol 19, 112-122. Lyon, M.A., Albertus, C., Birkinbine, J. & Naibi, J. (1996). A validity study of the social skills rating system-teacher version with disabled and non-disabled preschool children. Perceptual and Motor Skills, Vol 83, 307-316. Martens, B.K. (1993). Social labeling, precision of measurement, and problem solving: Key issues in the assessment of children's emotional problems. School Psychology Review, Vol 22, 308-312. McCalley, & Potter, (2002). Personal Communication, August 2002


55

McGuire, J. & Richman, N. (1987) Outcome of behaviour problems in the preschool setting. Child Care, Health and Development; 13:403-414. McSherry, J. (2001). Challenging behaviours in mainstream schools. Practical strategies for effective intervention and reintegration. David Fulton Publishers, London. Mental Health Foundation 2002 www.Mentalhealth.org.uk/ html/content/mentalhealth.cfm Merrell, K.W. (1993). Using behavior rating-scales to assess social skills and anti-social behavior in school settings – Development of the School Social-Behavior Scales. School Psychology Review, Vol 22, 115-133. Merrell, K. W. (2003). School Social Behavior Scales (2nd ed.). Assessment-Intervention Resources (www.assessment-intervention.com). Merrell, K. W., & Caldarella, P. (2003). Home & Community Social Behavior Scales. Assessment-Intervention Resources (www.assessment-intervention.com). Merrell, K.W., Streeter, A.L., Boelter, E.W., Caldarella, P. & Gentry, A. (2001). Validity of the Home and Community Social Behavior Scales comparisons with five behavior-rating scales. Psychology in the Schools, Vol 38, 313-325. Mitchell-Copeland, J., Denham, S.A., & DeMulder, E.K. (1997). Q-sort assessment of child-teacher attachment relationships and social competence in the preschool. Early Education and Development, Vol 8, 27-39. Morris, E. (2001). Self-Esteem Indicator. http://www.schoolofemotional-literacy.com Offord, D., Janus ,M. and Walsh, C. (2001) Population-level assessment of Readiness to learn at school for five year olds in Canada. Ontario The Canadian Centre for the Study of Children at Risk, McMaster University Pascal, C. & Bertram, T. (2001). The AcE Project. Accounting Early for Life Long Learning. Centre for Research In Early Childhood Education, Worcester. Priest, S. (2000) Personal Communication. QCA (2002) http://www.qca.org.uk/ca/foundation/foundation_stage_profile.asp Riding, R., Rayner, S., Morris, S., Grimley, M. & Adams, D. (2002). Emotional and Behavioural Development Scales. Birmingham, UK: Assessment Research Unit, School of Education, University of Birmingham. Roberts, W., Hom, A. & Battistich, V. (1995). Assessing students’ and teachers’ sense of the school as a caring community. Presented at The American Educational Research Association, San Francisco, USA, April 1995.

http://www.assessment-intervention.com/�

http://www.schoolofemotional-literacy.com/�

http://www.qca.org.uk/ca/foundation/foundation_stage_profile.asp�

56

Roche, A. (2002). Personal Communication, June 2002. Saarni, C. (1997). Emotional competence and self-regulation in childhood. In P. Salovey, P. & Mayer, J.D. (1990). Emotional intelligence. Imagination, Cognition and Personality, Vol 9, 185-211. Salovey, P. and Sluyter, D.J. (1997) (Eds.) Emotional Development and Emotional Intelligence, (p 10). Basic Books, New York. Seligman, & Rockwell, (2002). Personal Communication, July 2002 Sharp, P. & Faupel, A. (2001). Promoting Emotional Literacy. Guidelines for Schools, Local Authorities, and Health Services. Southampton, Southampton City Council Emotional Literacy Interest Group (SELIG) Solomon, D., Battistich, V., Watson, M., Schaps, E. & Lewis, C. (2000). A six district study of educational change: Direct and mediated effects of the Child Development Project. Social Psychology of Education, Vol 4, 3-51. St. James-Roberts, I., Singh, G., Lynn, R., & Jackson, S. (1994). Assessing emotional and behavioural problems in reception class school children: Factor structure, convergence and prevalence using PBCL. British Journal of Educational Psychology, Vol 64, 105-118. Steiner, C. (1997). Achieving Emotional Literacy. Bloomsbury Publishing plc, London. Tew, M. (2002). Personal Communication, May 2002. Topping (1998). ‘Promoting social competence’ http://www.dundee.ac.uk/psychology/prosoc.html Valla, J.P., Bergeron, L., & Smolla, N. (2000). The Dominic-R: a pictorial interview for 6 to 11 year-old children. Journal of the American Academy of Child and Adolescent Psychiatry, Vol 39, 85-93. Waters, E., Goldfield, S. & Hopkins, S. (2002). Indicators for child health, development and wellbeing. A systematic review of the literature and recommendations for population monitoring. Centre for Community Child Health, Royal Children’s Hospital, Melbourne. Weare, K. (2002). What works in developing children’s social and emotional competence and well-being. London, DfES Weare, K. (2000 ). Promoting Mental Social and Emotional Health: A whole school approach. London Routledge.

http://www.dundee.ac.uk/psychology/prosoc.html�

57

Wing, L., Leekam, S.R., Libby, S.J., Gould, J. & Larcombe, M. (2002). The Diagnostic Interview for Social and Communication Disorders: background, inter-rater reliability and clinical use. Journal of Child Psychology and Psychiatry, Vol 43, 307-325. Wright, J.C., Binney, V., & Smith, P.K. (1995). Security of attachment in 8-12 year-olds: a revised version of the Separation Anxiety Test, its psychometric properties and clinical interpretation. Journal of Child Psychology and Psychiatry, Vol 36, 757-774.

58

Appendix I Methods Supplement Data Extraction Sheet for Research Paper Name of Measure Contents Type of measure Age group Population Who completes and how

Setting(s) Country Time to complete Development Authors How developed Validity Reliability When should children be assessed?/context?

Samples tested on Who should conduct them?

What are the barriers?:

Significance of practitioner as the assessor

Training? Issues for high-risk groups?

Views of schools/ practitioners and parents

Child’s Voice Purpose and benefits

Refs for studies using it

Comments Suitability for purpose

Sample Suitability Components of instruments for table

59

60

Appendix II Tables of Instruments Part 1: Instruments identified by literature review Table 1a: Description of the instruments Table 1b: Practical application of the instruments Table 1c: Summary of the instruments (NB the figures quoted in the tables are from the studies cited and not necessarily from the original validation studies of instruments.) Part 2: Instruments identified through contacts Table 2a: Description of the instruments/materials Table 2b: Practical application of the instruments Table 2c: Summary of the instruments Part 3: Instruments in development Table 3a: Description of the instruments

Table 3b: Practical application of the instruments Table 3c: Summary of the instruments

61

1. Instruments identified by literature review Table 1a: Description of the instruments (information derived from papers identified) Instrument Description Development Reliability Validity Behavioral and Emotional Rating Scale BERS Epstein et al., 2002 Epstein 1999 Epstein and Sharma 1997

52 items in 5 factors: 1 interpersonal strengths (14 items) measures child’s ability to control emotions or behaviours in social situations; 2 family involvement (10 items) child’s participation and relationship with family; 3 intra-personal strengths (11 items) child’s outlook on competence or accomplishments; 4 school functioning (9 items) competence in school; 5 affective strengths (7 items) ability to give and receive affection.

Content development was by random selection of 410 parents and health professionals from a list of attendees from a Mental Health research conference. 250 responded and provided 1200 statements about emotional and behavioural strengths. These were reduced to 15 categories, which were sent to 800 relevant professionals nationwide resulting in a list of 127 items. This preliminary instrument was piloted and reduced to 80 items. A second pilot reduced it further to 68 items. 68 items were subjected to factor analysis, 16 were found to be redundant and the remaining 52 were re-analysed and 5 factors resulted.

Internal consistency was very good (alphas for the 5 subscales and overall strength ranged from 0.79 to 0.99.). Test-retest: (n = 59) correlations all greater than 0.80 with over half above 0.90. Results of a similar magnitude were found for the inter-rater reliability tests (n = 96).

Criterion validity was assessed by comparing the performance of the normal children with the Emotional Behavioural Disorders children using the BERS. The expectation was that scores for each group would be significantly different. The two groups were compared in 6 ways and each test showed significant differences at the p=0.01 level.

Bullying-Behaviour Scale and Peer Victim Scale BBS and PVS Austin & Joseph, 1996

Two, 6-item self-report scales to assess bully/victim problems at school. These were mixed into Harter’s Self-Perception Profile for Children (S-PPC). This disguises the subtlety of measuring bully-victim issues in the classroom with instruments that are specifically about bullying.

PVS was developed by Neary and Joseph in 1994 and matches the forced choice format of the S-PPC, with three questions on (negative) physical and three on verbal victim attention. These were based on behaviours identified by Olweus (1993) in a sample of 12-18 year-olds. PVS was tested with 60 Irish school girls. BBS added and repeated with 120 boys and girls in N Ireland.

Internal reliability was satisfactory (PVS alpha=.83 and BBS =.82). Used the NI study with 120 children to confirm internal reliability.

PVS and BBS were associated with low scores on the rest of the S-PPC. Convergent validity: compared well with self and peer reports of bullying. PVS and BBS compared with the Birleson Depression Inventory (1981), both scales associated with higher scores on BDI.

62

Instrument Description Development Reliability Validity Child Behaviour Check List CBCL (Achenbach, 1991, Achenbach & Rescorta, 2001) Costenbader & Keller, 1990

Two sections: behavioural problems (118 items rated 0=not true, 1 = somewhat or sometimes true, 2 =very true often true) and social competency (20 items) inventory (child’s participation in extra-curricular activities, involvement with friends and academic performance). 2 global subscales are obtained for internalising (over-controlled) and externalising (under-controlled). 7 Narrow subscales include aggressive behaviour, anxious/depressed, delinquent, social problems, somatic complaints, thought problems, withdrawn.

This study compared three groups of children on two scales (CBCL and Conners Behaviour Scales for both parents and teachers). Both scales differentiated between referred and non-referred children. Concluded that both parents and teachers make a valuable contribution to assessing children.

Details of reliability tests in manual: good to excellent. Test-retest = .90

Details of validity tests in manual: good to excellent.

Child Behavior Checklist/1½-5 CBCL/1½-5 Caregiver-Teacher Report Form C-TRF Achenbach & Rescorta, 2000

99 problem items plus descriptions of problems, disabilities, what concerns parents most about their child, and the best things about the child

In manual. Manual details extensive data on standardization, norms, and reliability.

Manual details validity studies. http://www.aseba.org/products/manuals.html

Child Behaviour CheckList – Teacher’s Report Form CBCL-TRF (Achenbach & Rescorta, 2001) Harris et al., 1993

Teacher Report Form similar to parent report form: Behavioural items (120) are similar, but adapted for school setting, uses same 0-2 rating system. Social section is referred as adaptive functioning and assesses the child’s appropriateness of behaviour, learning and happiness.

CBCL used to identify emotional and behavioural problems in ordinary primary schools. There was a high level of agreement between teachers for referred and non-referred student status and subsequent classification using the CBCL criteria for psychiatric disturbance.

Internal consistency for problem behaviours section for both CBCL and TRF in manuals is v high .9+. Test-retest reliability from 1 week to 4 months: .64 to .89. Inter-rater reliabilities high. CBCL showed high levels of sensitivity and specificity.

Broad band scales were sig correlated with most of the narrow band scales confirming the validity of these. Discriminant validity: non-referred sig different from referred on nearly all scales.

63

Instrument Description Development Reliability Validity Child Behaviour Rating Scale CBRS Broder et al., 2001

18-item checklist used to rate adaptive behaviour and social competency. Uses a 5-point Likert scale: 1=never to 5=always.

Scale originally developed in 1985, but additional items added based on interview data from parents and teachers.

Test-retest 18/99 teachers asked to complete the rating form again 3 weeks later – strong relationship.

Factor analysis conducted and then confirmatory factor analysis resulted in 4 factors (self-maintenance: alpha =.79; social adaptation: alpha =.66; independence: alpha =.78; peer teasing: alpha =.86 (results for control groups reported here). The 4 factors and total score were compared between the experimental (facial abnormalities) group and the controls. Statistically sig diffs between two groups on total score and teasing (controls higher scores)

Child Development Program (Student Questionnaire) CDP Solomon et al., 2000 Roberts et al 1995 [CDP covers two instruments (CDP: child and teacher)]

CDP Student Questionnaire has 38 items and assesses perceptions of collaborative and supportive relationships among students, positive relations between students and teachers, closeness and intimacy, student participation and influence and ethnic caring. Attitudinal items use a 5-point Likert scale and behavioural items use a 5-point frequency scale. Teacher version has 15 items and measures perceptions of collaborative and supportive relationships among staff, closeness, teacher participation and influence and shared goals and values (uses a 5-point Likert scale: 1=strongly disagree to 5=strongly agree).

Builds on the work of Battistich and part of a 4-year investigation into schools as communities. Based on observations of behaviours in the classroom. This paper aims to describe the school as a community as perceived by students and teachers. Approximately one third of the questions were phrased negatively to control for response set. Nine student items (was 47) and two teacher items dropped due to lack of differentiation.

CDP student final measure had high internal consistency: alpha .91. and .89 for teachers.

Construct validity: factor analysis resulted in 3 factors: collaboration, helping and closeness; student influence; positive interpersonal relations. Teachers resulted in 2 factors: collegiality and shared goals and values; influence and participation in decision making. Convergent validity: between students and teachers was .35 (schools where teachers feel a strong sense of community are not necessarily places where children do. Levels of commonality (similarities) are about 15% between schools and between classrooms within a school.)

64

Instrument Description Development Reliability Validity Conners’ Teacher Rating Scale Revised Short Form CTRS-28 Fantuzzo et al., 2001 .

Shortened version of the Conners’ Teacher Rating Scale. 28 items designed to assess children from 3-17 years. 3 factors; conduct problems, hyperactivity and inattentive/passive probs.

Construct: Miller et al. study mid-80s: 400 pre-school, white middle class children in NY. Exploratory factor analysis showed CTRS-28 was appropriate for use with pre-schoolers. However no cross validation, and used principal component analysis which inflates factor loadings with less than 30 items. + no concurrent validity.

Teacher version internal consistency .92,.91 and .89, for parents .84,.81 and .74. Performed same statistical analyses as for PIPPS.

Similar to other Fantuzzo paper. Teacher and Parent versions of the PIPPS (32 items, see below) were used. Teachers report play in school and classroom, parents play at home and neighbourhood. 3 factors; play interaction, disruption and disconnection. Exploratory factor analyses and then construct validity repeat of process to check factor structure in Miller data. Convergent validity: compared behaviours reported in home and school using CRTS-28. 3 factor structure most appropriate solution: conduct, hyperactivity and passivity. Internal consistency .94, .92 and .75. + confirmatory factor analyses. Showed sig structural congruence between this and Miller data. Concurrent: multi source PIPPS, Q-sort Convergent and divergent validity: compared well with PIPPS

Diagnostic Interview for Social and Communication Disorders DISCO Wing et al., 2002

Diagnostic interview for autism. Systematically obtains information about child’s clinical history from birth.

DISCO is a development of the Autism Diagnostic Interview ADI developed in 1989 (Rutter and colleagues: Le Couteur et al., 1989) which is diagnostic and provides a health history from birth. DISCO different: includes wider range of developmental domains including self-care, emotional disturbance and more

Inter-rater study: DISCO 9, results led to version 10, currently in use. Each item was rated numerically. 3 rating scales, based on types (a-c) of developmental function: current level of functioning, concerned delay, and untypical behaviour. Type a: steps identified and

65

Diagnostic Interview for Social and Communication Disorders DISCO cont.

pathological aspects. Items selected relevant for social interaction and communication. Developmental differences in domains obtained using Vineland Adaptive Behaviour Scales (Sparrow et al., 1984). + items derived from clinical experience (these domains are main reason for breadth of DISCO and were included to obtain understanding of problems experienced by children). Further development for infantile autism and Asperger syndrome.

ordered chronologically; type b: marked as delay, minor delay or no problem; type c scored twice: never and current, + severe, minor, not present. Both interviewers present at each interview. Total of 508 ratings and where there were fewer than 12 children, item was excluded. Agreement: Cohen’s Kappa + intra-class correlation > .75. Inter-rater analyses: 1 agreement on which items excluded; 2 agreement on those included. Agreement overall was 85.2% for school age and 85.0% for pre-school.

Dominic-R: A Pictorial Interview Valla et al., 2000

Pictorial interview-based questionnaire designed to assess mental disorders in 6-11 year-olds. Series of pictures (of Dominic, who can be male or female) in varying situations. Interviewer reads out statements to elicit replies. 2 booklets (49 and 47 pages, first has 4 pages of instructions). A version for African American children, ‘Terry’, exists.. Diagnoses: simple phobias, separation anxiety, over-anxiouness, depression, ADHD, and conduct disorders.

Earliest version was in 1981, developed as short screening tool and had 26 pictures. By 1994 it contained 194 drawings and comprehension of the tool was tested with 150 primary school children, drawing by drawing. 2 judges decided if any one drawing would be retained and the final instrument was put through the assessment procedure again. Questions were added to reduce any ambiguity in children’s reactions to the drawings. Also translated into Spanish, German and French and there is a computerised version.

Test-retest reliability at one week: coefficients ranged from .60+ for 21 symptoms and .50 and .59 for 50 symptoms. Internal consistency for the 7 mental health diagnoses ranged from .64 to .83.

Concurrent validity against the Schedule for Affective Disorders and Schizophrenia for School-Age Children, ranged from .39 to .56.

66

Instrument Description Development Reliability Validity Emotional Instability, Prosocial Behaviour and Aggression Scales EIPBAS Caprara & Pastorelli, 1993

Three scales: emotional instability (EI), prosocial behaviour (PB) and aggression (A). Ratings from 4 sources (2 adult and 2 child). EI: 20 items (scored often=3; sometimes=2; never=1) capacity to refrain from impulsivity and emotionality. PB: 15 items, same format, denoting altruism, trust and agreeableness. A: 20 items, same format, behaviours aimed at hurting others physically and verbally.

Literature review and based on theories of control. Authors identified 3 constructs that make up the instrument. Peer version developed from high loading items from the child version. Teachers and Parents version, similar with modified questions.

Internal consistency: 3 scales and for 3 groups of respondents, consistencies ranged form 0.97 to .099.

Concurrent validity: used the Teacher Report Form and the Parent Form from the CBCL. Teachers: EI and A showed a positive relationship with external subscale from the CBCL and PB was negatively related to internal and external subscales. Mothers EI, PB and A were positively related to internal and external subscales.

FOCAL Mitchell-Copeland et al., 1997 FOCAL cont.

Observational assessment. Children are observed for 5 min on 12 occasions. Children’s emotions and reactions to peers emotions observed during free play. Recorded on computer.

Designed to test the attachment of young children to their mother and their teacher. Home visitors observed mother-child attachment for a minimum of 2 hours and observed children in free play at school. Observers record event-based emotional interchanges using a validated naturalistic coding scheme and data were collected via computer. The derived outcome measures form two scales of social competence: overall positivity (emotional displays coded: happy, sad, angry, afraid, tender, hurt, neutral and other), the second scale, reactions to peers emotions, coded positive reinforcement, positive, negative and opposite affect match, helping, comforting/questioning, looking, ignoring, sad/hurt feelings, anti-social and negative

FOCAL two-hour video-tapes watched by researchers. Composite reliability, correlating the sorts of all the observers ranged between .68 and .89 across 7 mother-child pairs.

FOCAL compared with Attachment Q-sort and a teacher rating scale in development (Social Competence and Behaviour Evaluation, SCBE-30). Aim was not to validate FOCAL but to see comparisons – results were not sig different from each other.

67

reinforcement. Overall positivity was scored thus: relative frequency of occurrence for each emotion was calculated by dividing the number of observations by the total affects observed. Reactions to peers was calculated by sub-tracking Z-scores for the relative frequency of negative reactions from Z-scores for the relative frequencies of positive reactions. Observed emotions listed in paper.

Home and Community Social Behaviour Scales HCSBS Merrell & Calderella, 2003 Merrell 2003 Merrell et al., 2001 Lund & Merrell, 2001

HCSBS is parent version of the School Social Behaviour Scales (Merrell, 1993). 2 scales: 32 item social competence scale and 33 item anti-social scale. All items are rated using a 5-point scale 1=never to 5=frequently.

HCSBS standardised with a nationally representative population (ethnically and geographically diverse) n=1858. Additional data sources about children’s behaviours broadens the scope of the instrument. This helps overcome one of the disadvantages of limited sources (i.e. teacher as only source).

Paper reports high internal consistency: .96 to .98. Discriminant validity: differentiates clearly between non- and at-risk children.

Three separate studies focusing on convergent and discriminant validity evidence for the HCSBS in comparison with four scales: the Social Skills Rating System. Conners Parent Rating Scale-Revised-Short Form and Child Behaviour Checklist. HCSBS Scale A, Social Competence: evidenced: strong positive correlations - social skills and adaptability, strong negative correlations - externalising behaviour problems, modest negative correlations - internalising and atypical behaviour problems. HCSBS Scale B. Antisocial Behaviour: evidenced strong positive correlations - externalising behaviour problems, modest positive correlations - internalising and atypical behaviour problems and strong negative correlations - social skills and adaptability.

68

Instrument Description Development Reliability Validity Infant and Toddler Social and Emotional Assessment ITSEA Briggs-Gowan & Carter, 1998 Carter et al,1999

Designed to assess multiple dimensions of social and emotional competence. 2 versions: 1. 140 items for research use, 2. 60 items as a screener. Not designed as a stand alone instrument. Rated on following scale 0=not true/rarely, 1=somewhat true/sometimes, 2=very true/often.

Developed with 214 parents (SES diverse). Recruited via a paediatric clinic. Piloted with small sample – no. not given. Parents found it very acceptable to complete. Items based on literature, clinical observations, existing checklists. + 15 experts reviewed an initial pool of 200 items for content validity and age appropriateness.

Results for long version. Test-retest study: 34 parents selected at random, 29 completed ITSEA twice, at 2 weeks and at one year. (n=70) as part of the follow up (results not given). Internal reliability good alphas – those scales less than 0.7 resulted in further development. 2 week re-test intra-class coefficient good .75 to .91, less good at one year .67 to .24.

ITSEA compared with three other measures (CBCL2/3, PSI/SF and CCTI-9) all these instruments have strong internal validity and good test-retest reliability. ITSEA externalising and internalising scales were sig correlated (.48 to .62 P<0.01). Confirmatory factor analysis – 9 problem scales and 7 competence scales. Needs larger sample to examine broad band effects e.g. externalising and internalising factors.

Interpersonal Competence Scale ICS-T Cairns et al., 1995

Rating scales. 18-items assessing social and behavioural characteristics of children and youths. Three factors: AGG (argues, fights), POP (popularity) and ACA (spelling, maths). Three subsidiary factors AFF (smile friendly), OLY (appearance, wins/achieves) and INT (shyness, worry). Uses a 7-point scale for each item. In use for 10 years.

Construction: reviewed literature for domains and tested them with 350 3rd and 4th grade students. Further analyses conducted with 695 children, 220 4th grade and 475 7th grade. This was part of the Carolina Longitudinal Study and so authors were able to follow children who had completed the ICS into adolescence.

Test-retest: short term 3 weeks, 58, 4th grade children + 7th grade study: median for scales was r = .89. Long term 1 year, median for scales was r = .42. Inter-rater: ranged from .80 to .88.

Factor analysis: AGG, POP and ACA emerged as distinct factors. Alphas were AGG: .81; POP: .71, ACA: .71, AFF: .67, OLY: .84. Robust developmental validity. It provides robust assessments of current behaviours and reliable predictions of future ones

Penn Interactive Peer Play Scale PIPPS Fantuzzo et al., 1995

Observations of coded behaviours form.

Head Start teachers and parents were involved in the development of instrument – enhance sensitivity to classroom and cultural variables as working with an African American population . Specific tactics children use to effect immediate sustainable play – ie play repertoires that children routinely displayed in free-play

Inter-rater reliabilities: for 100 children, 20 teachers compared with teaching assistants - .88 P<.001. Alpha’s for 3 factors (.9,.89,.9).

Factor analyses – 3 factors: disruption, disconnection and peer interactions. Disruption (aggression, anti-social behaviours) Disconnection (non-participation, hovering, withdrawing, wandering and being ignored). Peer interaction (sharing ideas, leading, helping

69

Penn Interactive Peer Play Scale PIPPS cont.

periods. 800 Head Start children were videoed, parents, researcher and teachers studied the highest 25 and lowest 25 ‘play active’ children – identify most salient behaviours that would distinguish high and low players. Adept players showed affective characteristics (active, animated and happy) less adept were disruptive, aggressive and easily frustrated or disconnected, quiet and withdrawn. Resulted in 36 Likert format scale items.

others and encouraging others to play) – lists in paper. Cross validation with random sub-samples: assessed degree of congruence .98. Checked to see items migrated but no item migrated confirming structure. Convergent and divergent validity: SSRS used 2 scales: social skills (pro-social behaviours rated according to frequency) and problem behaviour (frequency checklist). Found expected patterns.

Penn State Worry Questionnaire for Children PSWQ-C Chorpita et al., 1997

PSWQ-C is an adaptation of the PSWQ: changed wording and reduced Likert scale options to 4. 14 Items scored from 0 to 3, giving a total score between 0 and 42, with higher scores indicating higher degree of worry.

Exploratory factor analysis: principal factor analysis yielded one factor. Two items loaded weakly and were dropped. Internal consistency: alpha = .89 for 14 items and was .81 for the 6-11 age group.

Found moderate to good fit for 1 factor solution for both genders and older age group, but Grade 1 dropped from younger age group as a formal worry process was considered not to be evident at this age.

Convergent validity: compared PSWQ-C with Children’s Depression Inventory and the Revised Children’s Manifest Anxiety Scale: found sig correlations with the expected outcomes, particularly CDI as this assesses negative affect rather than depression.

Preschool Behaviour Checklist PBCL McGuire & Richman, 1988 St James-Roberts et al., 1994

22 items, standardised for 2-5 year-olds. Includes four main areas of behaviour: conduct problems, emotional difficulties, social relations and concentration. Each area has 3-4 items, which are scored 0=absence of problem, 1=possible problem 2=definite problem.

Two studies using slightly different procedures, to consider whether findings in one group of children and teachers were replicated in another. Factor structure: found 6 with a strong first factor of conduct/aggression, then social withdrawal, and emotional/sensitivity. Factors 4-6 were less clear and were different in the two studies. They suggest a cut-off score of 10 (McGuire and Richman suggested 12).

Internal consistency: alpha .83. Test retest after two weeks: .88. Inter-rater reliability: 83%. PBCL's factoral structure confirms that, in the reception class context, it measures emotional and behavioural variables in children as these are observed by teachers. Findings very similar to McGuire and Richman.

Construct validity found six factors. Convergent validity compared with the Preschool Behaviour checklist was .89. Two studies resulted in similar factor structures for three of the factors but remaining factors were more variable.

70

Instrument Description Development Reliability Validity Separation and Anxiety Test SAT Duffy and Fell, 1999, Wright et al., 1995

Interview with photographs. It assumes that responses to the fictitious situations in photographs will be analogous to responses to real situations. Three subscales: attachment, self-reliance and avoidance. Examples of questions given.

Based on Attachment Theory. SAT extended and validated by Wright et al 1995 and assesses responses of children to imaginary separation from parents. Needs training to interpret data – researchers showed high levels of agreement for category labels, over 90% for three scales. Complicated and time-consuming scoring system. SAT identified differences between ‘self’ and ‘other’ responses on attachment and avoidance scales but not self-reliance.

1. Test-retest was positive but low across all scales indicating a lack of stability of responses over time. Inter-rater reliability high. 2. Inter-rater was acceptable (67% – 89%). Test-retest only conducted with clinical sample. Internal consistency was acceptable (alphas ~.70).

1. Discriminant validity: SAT differentiates between clinical and non-clinical subjects for emotional disturbance. 2. Similar findings.

Social Phobia and Anxiety Inventory for Children SPAI-C Beidel et al., 1995

Inventory is a self-completion instrument with 26 items, 18 of which are multiple answer. These rated on a 3 point scale: 0=never or hardly ever, 1=sometimes, 2=almost always or always. Scores are out of 52.

Items generated empirically and subjected to a two stage construction strategy. Items came from 1. clinical interviews with 20 socially phobic children (mean age 10.6 years); 2. daily diaries of these children were reviewed; 3. items found in the adult version of the SPAI were reviewed. These items were reviewed by 6 experts and the initial version contained 32 items. Item reduction phase: 20 different phobic children and 33 normal children – 6 items were dropped from the inventory because they did not differentiate between the children, hence 26 items. This version tested again in a phobic population. Also tested inventory for normative data. Combined studies: 52 phobic children and 48 normal.

154 children (70 phobic interviewed as before, 33 normal) Children completed the State Trait Anxiety IC and FSSC-R; parents completed the CBCL. This study was used for the reliability and validity tests. High test-retest reliability based on interviews using the Anxiety Disorders Interview Schedule for children. Alpha=.95. Two weeks after initial questionnaire, 62/154 completed the SPAI-C again: r =.86 P<.001. Also good at 10 months. Scale reliable for children as young as 8 years. Kappa: diagnostic classification social phobia was .87 and .83 for overanxious disorder.

Concurrent and external validity showed statistically sig correlations with other self-report measures of general anxiety. Factor analysis: 154 children used best fit solution and retained items with factor loadings greater than .45. Identified 3 factors; assertiveness/general conversation, traditional social encounters and public performance. Normal sig dif on all 3 factors. 38/154 r =.5 P<.001 SPAI-C compared with CBCL 74/154, got good correlations in the expected pattern e.g. inverse relationship between anxiety and social competence.

71

Instrument Description Development Reliability Validity School Social Behaviour Scales SSBS Merrell, 1993 Emerson et al., 1994 Merrell 2003

Teachers rate children on a 1-5 scale for each item in each subscale. Scale total scores then converted into a standard score and percentile ranks (from normative scores in the manual) + raw scores from each subscale can be converted to normative social functioning levels.

Developed to capture strength and sophistication of problem behaviours, but with more accent on social competence than the Teacher Rating Form. + made more user friendly than previous scales. Half the length of TRF and specific to school settings. Designed for K – 12 two scales: social competence (interpersonal skills, self-management skills, academic skills) and anti social behaviour (hostile-irritable, anti-social-aggressive, demanding-disruptive). 1858 students grades K – 12 from 22 school districts (range of SES, urban/rural, ethnic etc.) (ethnicity not major influence on behaviour but gender is, girls found to be more socially competent).

1. Reliability high: alphas: .94 to .98 for 2 summed scales and 6 sub-scales. Test-retest: .6 to .83 for same 8 scales. Inter-rater: .53 to .83 (weakest on anti-social behaviours). 2. Internal consistent reliability high: .88 for anti-social behaviour and .91 for social competence. Results for this sample similar to manual.

1. SSBS compared with four other validated measures: content, construct and discriminant validity all high. 2. Subscale correlations high: SC r=.78 to .84; AB r=.77 to .84. Compared SSBS with CBCL and TRF and CBCL. Found expected patterns of results: high r=.73 between SSBS total social competence and TRF adaptive functioning etc. Authors felt the convergent validity good enough to support use of SSBS to assess social behaviour in the school setting.

Social Ability Braza et al., 1993

Observation of 2 indices: amplitude of behaviour and amplitude of partnership. Children scored on on the basis of diversity of social contacts and behaviour.

Children were videoed twice a week for 30 minutes of free-play and were unaware of observers. Used Shannon’s Index to assess the relative frequency of time that each individual spent with different peers and to assess the time devoted to different social behaviours. 22 social patterns were observed and were grouped into 8 variables based on motivation.

Inter-observer reliability: 89.7%.

Factor analysis: revealed a 4 factor solution: ‘prosocial’, ‘agonistic’, ‘assertiveness’ and ‘prohierachy’ (acknowledging hierarchies/rules). Both the indexes of amplitude of partnership and the amplitude of behaviour showed a significant dependence with 4 factors taken as independent variables.

72

Instrument Description Development Reliability Validity Social Skills Rating Scale SSRS-T Gresham & Elliot, 1990 Elliot et al., 1993 Social Skills Rating Scale – Teacher Version Lyon, 1996 [SSRS cover two instruments: teacher and parent]

3 major scales : social skills, problem behaviours (both 3-point Likert scales) and academic competence (5-point Likert scale). For preschoolers: 40 items (30 prosocial and 10 problem behaviour). Parent form 2 major scales (social skills and problem behaviours) and has 55 items on four social skills: co-operation, assertion, self-control and responsibility. Student version for high school.

Norm referenced and nationally standardised cross-informant rating scale. Includes a brief problem behaviour screening scale.

Reliability reported as adequate to excellent. Internal consistency for preschool: for all scales it ranged from .79 to .83. Test-retest: all scales .84 to .93.

Validity reported as adequate to excellent. No extra details for preschoolers

Strengths and Difficulties Questionnaire SDQ Goodman, 1997 Goodman 1994

SDQ is a brief questionnaire of social behaviours completed by teachers and parents with a similar version for self-report, for 3-16 year-olds. Five dimensions: conduct problems, emotional symptoms, hyperactivity, peer relations and pro-social behaviour with five items in each. 3-point rating scale: not true = 0, somewhat true = 1, certainly true = 2. Scores 0-40 for difficulties and 0-10 for strengths.

Extra items added to the Rutter Health and Behaviour Checklist and factor analysis to guide development. The SDQ has two scales (peer problems and pro-social behaviour) in addition to the Rutter.

Reliability was generally satisfactory (mean coefficient a: .73), inter-rater reliability (mean: 0.34), or retest stability after 4 to 6 months (mean: 0.62). Used receiver operating characteristic ROC curves to distinguish between high and low risk samples.

The predicted five-factor structure (emotional, conduct, hyperactivity-inattention, peer, prosocial) was confirmed. Internalising and externalising scales were relatively ‘uncontaminated’ by one another. SDQ compared with Rutter’s questionnaires for parents and teachers. Very good concurrent validity. ROC analyses showed good predictive validity for both measures as they could distinguish between the two samples.

73

Table 1b: Practical application of the instruments (these applications were indicated in the papers, instruments may have other applications not covered by research papers accessed) Instrument Type Age group Who completes & Time Setting Population in paper Behavioral and Emotional Rating Scale BERS Epstein et al., 2002

Rating scale 5-18 years Teachers, parents School US, 2,176 normative group and 861 EBD children, from 32 states.

Bullying-Behaviour Scale and Peer Victim Scale, BBS and PVS Austin & Joseph, 1996

Questionnaire 8-11 years Children, but not told about the bullying and victim aspects

Classroom UK, 425 (204 boys) general population from Merseyside

Child Behaviour Check List CBCL (Achenbach, 1991; Achenbach & Rescorta, 2001) Costenbader &Keller, 1990

Rating scale 6-18 years Parents ~20 mins

Home 1991: US 2,368 nationally representative, non-referred children. 2001: 1,753 nationally representative, non-referred children. US 80 non-referred children

Child Behavior Checklist/1½-5 CBCL/1½-5 Caregiver-Teacher Report Form C-TRF Achenbach & Rescorta, 2000

Rating scale 1.5-5 years Teachers, caregivers School, nursery

US 1,728 children US 1,113 children

Child behaviour check list – Teacher’s Report Form CBCL-TRF (Achenbach, 1991; Achenbach & Rescorta, 2001) Harris et al. 1993

Rating scale 5-11 years Teachers ~20 mins

School 1991: US 1,391 students 2001: 2,319 students UK 183 non-referred (39 girls) children

Child Behaviour Rating Scale CBRS Broder et al., 2001

Rating scale 5-18 years Teachers School US, K – Grade 12. 99 with facial abnormalities and 99 controls

74

Instrument Type Age group Who completes & Time Setting Population in paper Child Development Program (Student Questionnaire) CDP, Solomon et al., 2000

Questionnaire 8-12 years Children and teachers School US, 4000+ children from 24 elementary schools in 6 different school districts across the US (and 550 teachers). Normative sample, heterogeneous for ethnicity and SES.

Conners’ Teacher Rating Scale Revised Short Form CTRS-28 Fantuzzo et al. 2001

Rating scale 3.5 to 5.5 years

Teachers School US, 580 low-income, urban, African American children from the Head Start programme. 51% male

Diagnostic Interview for Social and Communication Disorders DISCO Wing et al., 2002

Semi-structured Interview with parents (clinician designed)

3-11 years Non-clinical researchers (+ 3 months training - described), systematic interview (interviewer makes judgments for rating each item. Instructions supplied but not wording – depends on family) Parents encouraged to bring photos, videos to interview – development. Interview info supplemented with teachers info, with any discrepancies in rating checked and observations. 3 hours

Home (for school age) University of Kent (for pre-school)

UK, 82 children: 50 school age [80 to 140 mo]; 32 pre-school [34 to 67 mo] across autistic spectrum [18 high functioning; 18 low; 17 learning disabilities; 13 specific language difficulties; 15 typical development.] recruitment: clinics and special schools typicals from local schools (SE UK)

Dominic-R: A Pictorial Interview Valla et al., 2000

Drawings with interview-based questions

6-11 years Trained lay interviewers (training takes 2-3 hours) 15-25 minutes

School Canada, 340 school lower-middle class children from 4 schools aged 6-11.

Emotional Instability, Prosocial Behaviour and Aggression Scales Caprara & Pastorelli, 1993

Rating scales 7-10 years Self-report, teacher, mother and peer nomination

Home school

Italy, 390 children (214 boys) age 7-10; 20 teachers. 291 mothers, and 390 peers (same children).

FOCAL Mitchell-Copeland et al., 1997

Observational measure

Pre-school Researchers Nursery US, 62 (31 boys) ages 34 to 56 months (mean 4.45 months) heterogeneous for ethnicity and SES.

75

Instrument Type Age group Who completes & Time Setting Population in paper Home and Community Social Behaviour Scales HCSBS Merrell & Calderella, 2003 Merrell et al., 2001 Lund & Merrell, 2001

Rating scale 5-18 years Parents and caretakers Home US, 3 Studies: 1: grades 6 and 7, n=127 (12 with learning difficulties). 2: grades 2 to 5, n=60 normal population. 3: 206 in-patient youths.

Infant and Toddler Social and Emotional Assessment ITSEA Briggs-Gowan & Carter, 1998 Carter 1999

Questionnaire 1–3 years Parents 40 +/-20 min

Home US, 214, (102 girls) 1-3years SES diverse pop. (214/345 – no diffs between parents taking part and those refusing). ITSEA compared with observations in a separate study with 1 year-olds.

Interpersonal Competence Scale ICS-T Cairns et al., 1995

Rating scale 8 years to adolescence

Teachers, parents 2-4 min

School, home

US, 4th and 7th grade children, general suburban middle class

Penn Interactive Peer Play Scale PIPPS Fantuzzo et al. 1995

Observational rating scale

3-5 years Teachers Classroom US, 312, African American children enrolled in Head Start – low income. 145 males

Pen State Worry Questionnaire for Children PSWQ-C, Chorpita et al., 1997

Self-report 6-18 years Children Classroom US, 199 children general population (109 girls)

Preschool Behaviour Checklist PBCL McGuire and Richman, 1988 James-Roberts et al., 1994

Rating scale 2-5 years Teachers Nursery UK, 23 teachers and 516 children and 35 teachers and 350 children

Separation and Anxiety Test SAT Duffy & Fell, 1999, Wright et al., 1995

Interview with photographs

8-12 years Children School 1. Ireland 13 non-clinical children (5 males). 2. England 21 non-clinical children

76

Instrument Type Age group Who completes & Time Setting Population in paper Social Phobia and Anxiety Inventory for Children SPAI-C Beidel et al., 1995

Rating scale 8-15 years, Children School US, Normal controls were 67% male mean age 10.1

School Social Behaviour Scales SSBS Merrell, 1993 Emerson et al., 1994

Rating scale Grades K-12

Teachers, counsellors, school psychologists 7-9 mins

School 1. US, 1,858 general population, with ethnic and SES heterogeneity. 2. US, 210, non-referred children (51.9% male)

Social Ability Braza et al., 1993

Videoed observations

5 years Researchers School Spain, 27 (20 girls) one group of pre-school children from a grammar school.

Social Skills Rating Scale SSRS Elliot et al., 1993 Social Skills Rating Scale – Teacher Version Lyon et al, 1996

Rating scales 5-18 years Preschool children

Teachers, parents, students 20 min Teachers

School, home School

US, 4,170 grades 3-10, normative sample US, 49, (23 girls) aged 3-5 years from lower-middle to upper-middle socio-economic groupings. 27 were non-disabled.

Strengths and Difficulties Questionnaire SDQ Goodman, 1997

Rating scale 4-16 years Researchers, clinicians and educationalists, parents, child

School, clinic, home

UK nationwide epidemiological sample of 10,438 British 5-15 year-olds obtained SDQs from 96% of parents, 70% of teachers, and 91% of 11-15 year-olds. 403 children from either a psychiatric or dental clinic (n=158) in London

77

2. Instruments identified through contacts Table 2a: Description of the instruments/materials Instrument Description Development Users Reliability & Validity Adaptive Social Behaviour Inventory Hogan et al., 1992

Teacher rating scale. 30 items with each item score as rarely or never, sometimes or almost always. Three scales: Express (13 items); Comply (10) and Disrupt (7). Measures cooperation/ conformity, peer sociability, anti-social or upset behaviour.

Developed on the basis of a review of items in instruments by child development specialists. Piloted with mothers who were interviewed.

Professor Kathy Sylva, EPPE Project, Institute of Education

Horgan et al., 1992 waiting for paper.

Boxall profile Bennathan & Boxall, 1998

A guide to effective intervention in the education of children with emotional and behavioural difficulties. 2 sections: developmental strands section and the diagnostic profile.

Diagnostic developmental profile and identification of needs. Structured observation in the classroom developed for teachers and classroom assistants in Inner London. Developed as part of a nurture group approach to assess need, plan intervention and measure progress. Observations from children over time, with deviant behaviours brought together in 8 sets. ILEA Research and Statistics Branch reviewed the profile and some items were modified.

Tracey Sanders Hampshire County Council Colin Gordon, Southend on Sea Borough Council

Evaluation by ILEA Research and Statistics Branch.

Cogs Ball, 2002

Page of drawn cogs for child to fill in, scoring 10 concepts for or from? self-assessment (each cog can score up to 10). Class exercise.

No details New Manton Primary School, Worksop

No details

Coping in School Scale CISS McSherry, 2001

Tool to assess readiness for reintegration into mainstream schooling. 2 versions: 2 long (8 sections) for teachers: self management of behaviour and self and others and short (5 sections) for specific behaviours: self awareness, confidence and organisation, attitude, learning and literacy skills. Scores are ratings from

Developed at the Institute of Education as a PhD. Items were developed through own experience and suggestions from EBD and main school teachers in a two phase process. Piloted in Sutton. Reintegration Readiness Scale developed into the CISS. 4 groups of pupils from EBD schools.

Used In Wandsworth as a reintegration and inclusion tool for 3 years. Used in primary school at transition to secondary school. Also used in other London authorities, Oxford, Croydon and Kent.

Validation study showed significant differences between EBD children and mainstream children.

78

1-4: 1=never to 4=almost always able to fulfil this criterion.

Instrument Description Development Users Reliability & Validity Devereux Early Childhood Assessment Programme DECA LeBuffe & Naglieri, 1998

Programme to support early social and emotional health. Technical manual, user’s guide, parent’s guide, classroom guide and observational journal. Assesses social and emotional competence using a 37 item checklist with each item scored on a 5-point scale: 0=never to 4=very frequently. There are 4 subscales: initiative, self-control, attachment and behaviour concerns. Same versions for teachers and parents.

Developed by the Devereux Foundation as part of a series of school assessment instruments. Assesses children in early years to prevent emotional and behavioural disorders by assessing their resilience and gauging protective factors. DECA was developed over a 2 year period. Factor analysis resulted in scales that were in keeping with research on protective factors: 3 protective factor scales: initiative, self-control and attachment, + behavioural concerns.

Internal consistency for each scale of the protective scales was >0.8. The behavioural concern scale was lower but reflecting the heterogeneity of the scale. Test-retest: over 24 hour period: teacher reliabilities for protective factors ranged .87 to .94 and parents ranged from .55 to .80. Inter-rater reliability between teachers for the protective factors range .59 to .77.

Criterion validity: predict if child was part of a clinical or non-referred sample. 69% were correctly classified. Construct validity was explored by comparing the protective factor scales with the behavioural concerns scale. Overall relationship was -.65, i.e. protective factors and problem behaviours are inversely related.

Early Development Instrument EDI Offord & Janus, 2001

A population based measure for communities. Assesses readiness to learn. Teacher rating scale with multiple answer and short questions. 5 domains: physical health, social competence, emotional health, cognitive development and communication skills.

Developed in conjunction with practitioners and the Early Years Action Group and the Parenting and Literacy Centres in Toronto.

Widely used in Canada Validity studies conducted and instrument found to be very satisfactory, with more studies in progress to ensure instrument retains its validity.

Early Years Profile, Hereford & Worcester Partnership, 1997

Profile completed on entry to full time education and is intended to reflect child’s attitudes, attributes and understanding. 30 items each addressed individually against

Used during the 90s in most H&W schools. Revised in 1990, 1995 and 1997 to balance needs of assessment and recording and the National Curriculum. (development via working party and Wolverhampton University)

Hereford & Worcester Development has been iterative therefore no formal reliability and validity studies. Inferred validity through revisions.

79

the manual criteria. Profile filled in using a colour code.

Instrument Description Development Users Reliability & Validity Emotional Behaviour Scale, EBS Clarbour & Roger, in press

Assess adolescent emotional coping strategies. 65 item self-report questionnaire. 3 subscales: social anxiety, malevolent aggression and social self-esteem.

Based on the emotional control model of one of the authors (Roger). Item construction was developed by asking for reactions from Year 7 pupils to a range of situations. They were asked how they would feel and how they thought others would feel. Exploratory factor analysis was conducted and failing items were rejected.

Jill Pattenden, Jane Clarbour, University of York

Internal reliability coefficients ranged from .76 to .90. Validated in school children (11-14 years) and prison populations. Concurrent validity with Harter’s SPP-C and Goodman’s SDQ, which provided confirmatory evidence for predictive value of EBS. Factor analysis yielded 3 factors

Emotional and Behavioural Development Scales, EBDS Riding et al., 2002

Brief checklist of 3 areas of emotional and development behaviours. 3 scales of 5 items rated on a 6-point scale 0=not at all to 5=always.

Survey of Principle Educational Psychologists and personnel in LEAs working in the EBD area. List of 250 emotional and behavioural descriptors was produced. These were grouped to give 21 items describing observable behaviours. The scales went through three revisions resulting a final version with 15 items.

Birmingham University, Assessment Research Unit, School of Education.

Factor analysis resulted in 3 factors: conduct, learning and emotional behaviour.

Emotional Quotient Inventory- Youth version (S) EQ-I:YV(S) Bar-On & Parker, 2000

30-item questionnaire mostly about emotional competence. Scored on a 4-point scale: 1 = not true of me to 4 = very much true of me. Short form takes 15 min. 5 scales: intrapersonal, interpersonal, adaptability, stress and total EQ.

Series of questionnaires measuring emotional intelligence. Adult, youth and child versions. Data were collected from 4 age groups 7-9 (n= 2601) and 10-12 (n=3144) years and two older groups. Development details are in a doctoral dissertation.

Geetu Orme is the director of Ei UK and the home website is www.mhs.org.

Reliabilities: internal consistencies: 0.65 to 0.81 (7-9 years); test-retest: 0.87. Well validated: factorial and construct. Part of a series of assessment instruments for measuring emotional quotient.

Enable Banks et al., 2001

Computer based. Reporter (teacher, parent or child) chooses statements that fit the behaviours of subject from a selection of topic areas. Then chooses 6 most relevant statements. Software identifies

Developed over a 10 year period in consultation with teachers and theory-based. Much of recent development has been in collaboration with Christchurch College Canterbury. Software identifies earliest stage of development at which subject has ‘missed out’ and suggests ways of repairing the gap in

Julia Bird, and three others (two ex-social workers and another ex-teacher; now all psycho-therapists)

Development has been iterative therefore no formal reliability and validity studies. Inferred validity through revisions. An external evaluation (funded by the DH) is planned for 6 UK settings.


http://www.mhs.org/�

80

strategies to repair missing developmental stages for child on personal level, in classroom and related to the Curriculum.

a manner that maintains the subject’s dignity. Instrument has evolved - responding to needs of teachers i.e. from a need to address negative behaviours to positive screening.

Instrument Description Development Users Reliability & Validity Fast Track Greenberg et al.

Fast Track has potential and is a multifaceted series of measures including child interview, parent and teacher rating scales. It is a comprehensive, multi-site intervention designed to prevent serious and chronic antisocial behaviour in a sample of children selected as high-risk at school entry because of their conduct problems in kindergarten and home. The intervention is guided by a developmental theory positing the interaction of multiple influences on the development of antisocial behaviour.

All development details at http://www.fasttrackproject.org Developmental model targeted at transitions at school entry and from elementary to middle school. The most intense phase of intervention took place in the first grade year for each of three successive cohorts. The six components of the first grade intervention include: 1. teacher-led classroom curricula (called PATHS) as a universal intervention directed toward the development of emotional concepts, social understanding, and self-control; 2. parent training groups designed to promote the development of positive family-school relationships; 3. home visits; 4.child social skill training groups; 5. child tutoring in reading; and 6. child friendship enhancement in the classroom.

Mark Greenberg Penn State University. Peter Appleton, Cambridge and Peterborough NHSS considering using Fast Track in future.

Series of technical reports about the development of all the instruments. Those who completed these assessments were included in the total sample based on the school that they entered for first grade. In this way three successive cohorts (yearly) were recruited to yield a total of 440 intervention and 440 control children.

Leuven Involvement Scale for Young Children LIS-YC Laevers, 1994

The scale assesses levels of involvement in learning. Two components: list of signals and 5 scale rates. Assessors have to be trained and so the manual also has a video that trains teachers to rate involvement on a scale from 1=no activity to 5=sustained intense activity. Individuals and classes can be assessed. Versions exist for other age groups including adults.

Underpinned by the theory of Experiential Education EXE. Action research revealed two indicators: emotional well being and involvement. The involvement scale is a process-oriented monitoring system (focuses on wellbeing and involvement). An instrument to let children from 5-12 map their own levels of wellbeing and involvement at school (in general and for up to 21 subject area’s). Includes action points to improve the environment. Also developed the Self-Monitoring System for trainee teachers.

In use since 1993, English, French, German and Spanish versions.

Reliability: inter-rater reliability was .9. Test retest showed that level of involvement was a relatively stable characteristic and is an interaction of the context (how the teacher handles the group) and the characteristics of the children.

http://www.fasttrackproject.org/�

81

Instrument Description Development Users Reliability & Validity Mary Layton MA, 1996

Structured interview: 13 vignettes that have a coded response (assertive = 3, passive constructive =2 passive unconstructive = 1, aggressive = 0) + 8 pictures (e.g. child fallen over in playground) with coded responses – again 3 to 0, positive to negative.

Designed to investigate prosocial skills: communication and social skills; level of awareness of others; application of positive ways of resolving disruption and conflict; positive choices – showing personal responsibility and power. Piloted with 4 children. Questionnaire had a balanced spread of emotional, moral and behavioural issues so that prosocial skills tested in a variety of situations

Evaluation of the School and Family Links Programme: aims to improve prosocial behaviours in 5-8 year-olds.

N/a

New Close Primary School Project McCalley & Potter, 2002

Check list for teachers and children on emotional intelligence. The self assessment part consists of 2 components: How I see myself. Children rank themselves (1=never to 4=always) on 11 statements; and 12 written statements that children complete (e.g. today I feel …)

Used Salovey’s 5 domains: knowing one’s emotions, managing emotions, motivating oneself, recognising emotions in others and handling relationships. Adapted interpretation of the National Curriculum to address the goals attached to each of these domains. Used multiple sources of information about children (behaviour books, bullying questionnaire, parental comments, interviews with children, staff comments, professional judgement, SATs scores etc). Also used an Attitudes to School questionnaire. Used all the information to plot a matrix and then devised a How I see myself questionnaire

Authors Evaluation conducted by authors, showed improvements in emotional intelligence and SATs performances.

The Optimistic Child Seligman & Rockwell, 1996

An assessment for children (ages 8-12). It measures the children’s explanatory style (optimism vs helplessness).

No details Martin Seligman/ Sylvia Rockwell UIC.EDU

Process-oriented Child Monitoring System POMS Laevers et al., no date

Three stages: 1 class screening (socio-emotional problems and developmental problems); 2 individual observation and analysis; 3 setting out goals for action. Under-pinning the

Developed at the Centre for Experiential Education in Leuven by Prof Laevers. Two strong indicators of quality of development from the Experiential Education Project were wellbeing and involvement. As with the LIS, teachers are asked to observe children and

Centre for Experiential Education

Validation study showed significant differences between EBD children and mainstream children.

82

Process-oriented Child Monitoring System (POMS) cont

assessment are the following concepts: Basic needs: physical needs, need for affection, warmth, tenderness, need for safety, clarity, continuity, need for recognition, need to experience oneself as competent, need for moral correctness and give life meaning. Signs of well-being: openness and receptivity, flexibility, self-confidence and self-esteem, able to defend oneself and assertiveness, vitality, relaxation, enjoyment without restraints, being in touch with one’s self.

rate them on scales. The scales have been developed with either 3 levels (variant A) or with 5 levels (variant B) for more differentiation.

Pupil Attitude to Self and School PASS Goodall, 2002

Computerised 50 item rating scale assessing children’s attitudes to themselves as learners and their school. 9 factors: 1: feelings about school; 2: perceived learning capability; 3: self-regard; 4: approach to learning situations; 5: attitude to teachers; 6: attitude to work; 7: confidence in learning; 8 attitude to attendance; 9: attitude to work demands.

Developed in collaboration with the Universities of Exeter and Birmingham. Principles underpinning PASS were social inclusion, ethical testing, ecological perspectives and pragmatism. Theoretical perspectives are based on Tuckman’s model (1999). Three versions developed and piloted independently in primary, secondary and FE sectors with representative samples (n>6,000). Evaluated.

Goodall, Cumbria LEA Item and factor analysis conducted indicated high reliability and validity.

Instrument Description Development Users Reliability & Validity Record of assessment for emotional literacy Mason Moor Primary School,

Checklist for 3 aspects of children’s awareness: handling relationships, managing and knowing own emotions and recognising emotions in other.

Part of the Southampton Emotional Literacy Interest Group.

Mason Moor Primary School, Southampton

Evaluated in the Spring Term 2000. On balance the evaluation was favourable with the youngest children experiencing some difficulty

83

Items scored on a 4 point scale a=consistently to d=very rarely.

in performing in the assessment.

Reintegration Readiness Scale (this is based on, but different from, McSherry’s scale) Doyle, 2002

Quantitative assessment tool that analyses behaviour: readiness to reintegrate and identifies areas where further development is needed. Not as broad as the Boxall Profile. 5 scales: self-control and behaviour management, social skills, self-awareness and confidence, learning skills and approach to learning. Child is rated against statements in each scale on one of 4 levels (rarely to always fulfils this criterion). + blank profile chart to monitor change.

Builds on nurture group principles. The Boxall profile identifies children who would benefit from taking part in a nurturing group. The Readiness Reintegration Scale identifies when the child can be returned to the main stream. Based also on Portage Early Education Programme, McSherry’s Reintegration Programme and the Early Learning Goals QCA. A numerical score is derived from the statements and a score of 70%+ of maximum score is considered to indicate readiness for reintegration.

Rebecca Doyle, Canterbury Infant School, Thetford

None

Self-Esteem Indicator Morris, 2001

Brief check-list of 22 items. 3 subscales: sense of personal power, sense of self and sense of belonging.

Developed by Elizabeth Morris, influenced by the work of Gottman, at Washington University.

Linda Hoggan, Simpson Children Centre Milton Keynes.

No details given.

Short Term Education and Pupil Support STEPS

Check list of 91 items. 3 subscales: learning, conduct and emotional behaviours. Positive and negative dimensions of each item. Scored on a 1-6 scale; 1=always to 6=most likely.

No details given. The construction of this instrument is different as each item has positive and negative aspects e.g. attentive and listens vs distracted and inattentive respectively. The scoring is on a 1-6 scale, with positive behaviours scoring 6-4 and higher frequencies of negative behaviours scoring 3-1.

Stockport Metropolitan Borough Council

No details given

84

Instrument Description Development Users Reliability & Validity The Taking Care Project Sheffield.

Interview based. Uses scenarios of bullying and domestic conflict to elicit feeling words and help seeking intentions. Ability to express emotions improved. Addressed through circle time, expressive drama, etc.

Children were interviewed before and after the sessions, using scenarios of bullying and domestic conflict, to elicit feeling words and help-seeking intentions. Teachers were asked to rate individual children and whole class before and after the sessions. Explored ability to express emotions and numbers of feelings words they used spontaneously. Found: words such as miserable, glad, surprised and jealous.

One of the Emotional Literacy Initiative projects. Sheffield.

Evaluation of the Year 2 and 4 classes showed that sessions were beneficial to children. Significant improvements in ability to express emotions and seek help across age and gender. Hunt & Crow, 2001

85

Table 2b: Practical application of the instruments Instrument Type Age group Who completes & Time Setting Population Adaptive Social Behaviour Inventory Hogan et al., 1992

Rating scale 3-5 years Teacher School

Boxall profile Bennathan & Boxall, 1998

Rating scale Primary Teacher School Inner London primary school children, no details of numbers in the manual.

Cogs Ball, 2002

Self-report Primary Child School

Coping in School Scale McSherry, 2001

Rating scale Late primary into secondary

Teacher and pupil School EBD children and mainstream children from Wandsworth and other authorities across the country (n= 10).

Devereux Early Childhood Assessment Programme LeBuffe & Naglieri, 1998,

Rating scales 2-5 years Teachers and parents School 2000 preschool children from 28 states. 51% boys. 25% from low income families and sample was ethnically mixed.

Early Development Instrument, Offord & Janus, 2001

Rating scale Kindergarten Teacher School Cited in a systematic review

Early Years Profile, Hereford & Worcester Partnership, 1997

Rating scale Early Years Teacher School Manual and supporting material all Early Years children in Hereford and Worcester.

Emotional Behaviour Scale Clarbour & Roger, 2000

Self-report 8-19 years Child School 228, Year 7, 222 Year 8 and 191 Year 9 pupils from a comprehensive school

Emotional and Behavioural Development Scales Riding et al., 2002

Rating scale Infant, Junior and secondary

Teacher 1-15 min average time 4 min

School 900 teachers from 60 EBD schools, 10MLD/SLD schools and 75 mainstream schools.

Emotional Quotient Inventory EQ-i:YV Bar-On & Parker, 2000

Self-report 7-18 years Youths and children (+ version for adults)

School, home

US normative data.

Enable Banks et al., 2001

Computerised statement selection

Any Teacher (about child or self), parent, child

School (usually)

Series of primary schools in Newham with ad hoc development in others over the course of 10 years.

Fasttrack Greenberg et al.,

Selection of instruments

Various Various School, home

Website with selection of instruments primarily concerned with behaviour and social conduct.

86

Instrument Type Age group Who completes & Time Setting Population Leuven Involvement Scale for Young Children Laevers, 1994

Observation and Rating scale

3-5 years Teachers School Widely used. Used to monitor the Effective Early Learning project in Worcester where 3000 adults were trained and 30,000 pre-school children were observed.

Mary Layton, 1996 Structured interview

5-8 years Child School, home

Part of the evaluation of the School and Family Links Programme. 60 children (30 each in two schools, 1 control and 1 experimental)

New Close Primary School Project McCalley & Potter, 2002

Rating scale and written statements

10-11 years Teacher and child School 31, Y6 children at New Close Primary School

The Optimistic Child Seligman & Rockwell, 2002

Self-report 8-12 years Child School No detail

Process-oriented Child Monitoring System Laevers et al., no date

Rating scale 4-5 years cases in manual

Teacher School Manual includes case studies of 4 and 5 year-olds.

Pupil Attitude to Self and School Goodall, 2002

Self-report 8-16 years Child School UK normative data. 10,000 in 70 schools and standardised for Y4 to Y11. Results are given as centile scores standardised for age (individual) or % maximum scores standardised for whole sample (school).

Record of assessment for emotional literacy Mason Moor Primary School

Rating scale 7-11 years Teacher School Mason Moor Primary school

Reintegration Readiness Scale Doyle, 2002

Rating scale Infants Teachers ‘quick’

School Author’s own Infant School.

Self-Esteem Indicator Morris, 2001

Rating scale Early years Teacher School No detail

STEPS Stockport Council

Rating scale Teacher School No detail

The Taking Care Project Sheffield.

Self-report 5 -11 years Teacher School Developed as part of Antidote

87

Section 3: Instruments in development Table 3a: Description of the instruments Instrument Description Development Developers Reliability & Validity Accounting Early for Life Long Learning AcE Project Pascal & Bertram, 2001

Scales for teachers to rate children. Specific observation sheets detailing indicators of attitudes, dispositions and emotional wellbeing (intention is to identify children who need support). Practice focused for improving the learning experience of 3-6 year-olds. Scales: emotional literacy (eg express and work with own emotions, empathy, express vitality, appreciate moral culture); empowerment (eg unafraid, confidence to explore, pride, purposeful); connectedness (eg attachment to people, sense of belonging); positive self-esteem (eg demonstrate confidence, respect own beliefs, resolve conflict, sense of humour, express values).

Builds on the Effective Early Learning Project (EEL) (which used the Child Involvement Scale and the Adult Engagement Scale). Working in conjunction with the Experiential Education (EXE) project in Belgium to develop scales. Work on attitudes and disposition scales carried out. Second pilot in about 15 settings in Autumn 2002. Influenced by the work of Ferre Laevers. Based on observations of children (staff need training).

Fiona Ramsden, University College Worcester. Working in collaboration with Birmingham, Bristol, Nottingham, and Worcester LEAs.

This project is set in the qualitative research paradigm, using case study and action research methods.

Ellis & Jennings, 2002

Pre school transfer form that will lead to entry profile covering the first half term and end with the Foundation Stage Profile.

Being developed in Somerset. Based on the curriculum guidance for Foundation Stage. Simple format, teacher friendly.

Sue Ellis and Julia Jennings Somerset County Council

88

Instrument Description Development Developers Reliability & Validity Effective Lifelong Learning Inventory ELLI Effective Learning Profile Broadfoot et al., 2002

65 items scored on a 5 point Likert scale: 1= almost never to 5=nearly always. Designed with emotional competences embedded in the items – has a learning focus.

In development at Bristol University with National Learning Foundation backing. Items generated from three sources: items from an instrument developed in school based action research; literature review and consultation with experts. The instrument is self-report, and most of work done with 7-18 year-olds (n=1604 in second pilot, n=180 in first). Version for KS1 – JELLI and one for adults.

Broadfoot, Caxton and Deakin-Crick, University of Bristol.

Factor analysis completed for pilot (n=180). Showed 7 factors but 3 are similar so next version will have 5. Validity and reliability studies on-going.

Emotional Literacy Audit (of organisational culture). ELA Antidote

Assesses how to foster emotional literacy at the organisational level in schools.

Emotional Literacy Initiative will run from 2001 to 2004 and key strategy is to develop ESI. Three schools in Newham, children aged from nursery to 16 years. Research Questions: how do individuals experience the emotional and social atmosphere of the whole organisation? What factors inhibit or facilitate individuals in processing their emotional and social experience? How can emotional literacy be fostered at an organisational level? Using questionnaires, interviews and focus groups.

Antidote

Feelings and Empathy Questionnaire Browning, 2002

Dissertation based on emotional literacy. Feelings vocabulary, empathy for and perceptions of others. 3 tasks: what other people think of them, developing a vocabulary, short story to empathise with.

Initial work done to develop the three tasks: virtually finished. Part of research included short questionnaire. 3 sections:1 what they thought other people thought of them; 2. write as many feeling words as you can; 3. story how would you feel, what would you do in their place?.

MEd student at Bristol, about to submit. Behaviour support teacher Wiltshire

Formal reliability and validity studies not conducted, but before and after results in school very positive.

89

Instrument Description Development Developers Reliability & Validity Simon Priest, 2002 No details of instrument

supplied. Rating scale for teachers to reflect on their own practice, early attempt at developing instrument. In order to be effective, this would need to be developed into an all round appraisal.

Mentioned an early and naïve attempt for use by groups of staff to prompt self-reflection and discussion. Goleman formulation (self-awareness, self-regulation, motivation, empathy and social skills).

Senior Educational Psychologist Rotherham Metropolitan Borough Council.

Untested

Sefton Council Action research project developing a framework for teaching emotional and social skills with 4-11 year-olds.

Pilot. Using experiences to plan activities across the curriculum.

Angela Walker, Educational Psychologist Sefton

Evaluation planned but not conducted.

Social Competence Test SCoT Laevers, 2002 (3 Masters students)

Student’s instrument: 12 video scenes displayed by puppets + 2 Mr Bean fragments of video. Individual children are shown videos which are stopped several times following a standardised procedure. Children are asked about the following: role taking at the level of 1. feelings and emotions, 2. perceptions and cognitions, 3. intentions and motives, + understanding behaviour, predicting behaviour and acting in a socially competent way. Children are scored on a 4-point scale based on categorisation of answers.

The SCoT has a process orientation to assessment. It is the next step in assessing emotional and social competence. Laevers’ work has included the Learning Involvement Scale and the Box Full of Feelings. Three Masters students have submitted a thesis on the development of an instrument to assess emotional and social competence

Ferre Laevers, Prof at Leuven University.

Preliminary factor analyses conducted with 4-7-year-olds. Reliability and validity studies being planned. Student’s instrument: high level of internal consistency (0.80 to 0.93), high correlation between the two parts of the test (0.70 to 0.83) for different ages. High reliability at the level of scoring (0.81 to 0.97). High correlation between age and results but there is a ceiling effect for the 8 year olds. High level of involvement. Sensitive to change as a result of intervention with the Box Full of Feelings materials.

Talkit Tew, 2002

Profiling young people providing a vocabulary and a ‘talking tool’ to be used by form tutors and year 7 pupils. Profiling tool that will be computerised and compatible with SIMS. Potential to

Study started with conventional assessment measures. Developing a personal construct framework for children – i.e. children develop their own framework and this is a method to assess it. Statements from

Marylin Tew, PhD student at Bristol

Derived 154 statements from interviews with children, these have been verified by triangulation (comparisons of children’s groupings of statements – if the same statements are grouped together by different children, this

90

compare self, peers, teacher, parent. Asks children to profile themselves emotionally in the context of the school 5 subscales: controlling emotions, keeping motivated, confidence in different situations, inter-personal empathy and a 5th smaller scale for integrity.

children – developing a talking tool.

indicates validity in the grouping).

What I think about my school Roche, 2002

Questionnaire for 7-10 year olds

Based on the QCA document, as NSPCC emotional assessment questionnaire, Healthy schools information, Cumbria Behaviour Curriculum and Birmingham frameworks for Interventions.

Paula Crosbie, Education Dept. Nottingham

None

91

Table 3b: Practical application of the instruments Instrument Type Age group Who completes & Time Setting Population in paper Accounting Early for Life Long Learning Project Pascall & Bertram, 2001

Observational scale

3-6 years Teachers, practitioners School UK children

Effective Lifelong Learning Inventory Broadfoot et al. 2002

Self-report 7-18 years Child School 180 in first pilot with 1604 in second pilot.

Ellis & Jennings, 2002 Rating scale Teacher School No details Emotional Literacy Audit (of organisational culture). Antidote

Questionnaire, interview

3-16 years Child, teachers School 3 schools in Newham

Feelings and Empathy Questionnaire Browning, 2002

Interactive interview/self report

7-11 years Teachers School 7-11 year olds in her study

Talkit Tew, 2002

Child interview Year 7 Child, peers, teacher, parent. School 11-12 year olds in author’s study, but could go down to 8 years and older than 12 years.

Sefton Council Rating scale 4-11 years Teacher School Pilot, no details of numbers Social Competence Test (Laevers, 2002)

Video based 4-8 years Children School, Master’s degree subjects (n=41)

What I think about my school Roche, 2002

Self-report 7-11 years Children School No details

Some of the instruments are in early development and so data were limited. The following instruments have not been included in Tables 3b and 3d because we had insufficient detail about them : (those developed by Ellis, Priest and Prince, the Inch Project, Play wise, and Sefton Council).

92

Documents

Assessing Emotional and Social Competence in Primary School and