http://www.tandf.co.uk/journals/pp/01650254.html

Development of childrens knowledge about the mentalworld

John H. FlavellStanford University, Stanford, CA, USA

This article is a survey of research on the childhood acquisition of knowledge about the mind,especiallyworkdone during the past two decades under theheading of theory-of-mind development.It begins with a history of research in this area. This is followed by a brief summary of principaltheories and ndings. The article concludes with some guesses about future research in this excitingarea of cognitive development.

Adevelopmental psychologist shows a 5-year-old a cookies boxwith a picture of cookies on it and asks her what is in it.Cookies, is the readyanswer. The child then looks inside thebox and to her surprise sees that it actually contains crayons,not cookies. What would another child who had not yetopened the box think was in it? the experimenter now asks.Cookies! says the child, amused at the trick. The experi-menter then tries the same procedure with a 3-year-old. Theanswer to the rst question is the expected cookies, but theresponse to the second is unexpected: crayons. Even moresurprising, the child alsomaintains that he himself had initiallythought that the boxwould contain crayons. Unlike the 5-year-old, the 3-year-old shows no evidence of understanding thateither he or other people could hold a belief that is false.

Results such as this are found in a currently ourishingresearch area concerning the development of our knowledgeand beliefs about the mental worldour folk psychology ornaive theory of mind. More than did earliermetacognitive andsocial-cognitive investigations in the same domain, thisapproach probes childrens conceptions of the most funda-mental components of the mind, such as beliefs and desires,and childrens knowledge of how these components affect andare affected by perceptual inputs and behavioural inputs. Inless than 20 years, this fast-growing area has spawnedhundreds of research articles and scores of book- andmonograph-length treatments. Indeed, the spate of papersand posters on this topic at recent meetings of the Society forResearch in Child Development reminded several olderparticipants of the way Piagetian research dominated theprogramme in years past. To illustrate, a recent meta-analysisof false belief studies alonejust one topic in this areaincluded 77 research articles, encompassing 177 separatestudies, and 591 false-belief conditions (Wellman, Gross, &Watson, 1999). Developmental ndings in this area have alsorecently become of interest to philosophers of mind, whobelieve that these ndings may help clarify philosophicaldisputes about the nature of folk psychology.

Why this intense research interest in the development ofknowledge about themental world?One answer is that human

social and cognitive life bereft of such knowledge seemsvirtually unimaginable, and that the development of somethingthat important is surely worth learning about. In her lectureson this topic, Alison Gopnik likes to make the point in thefollowing way: Imaginewhat it would be like for you to give alecture to an audience if you had no conception of mentalstates. The audience might appear to you as bags of meat withtwo small holes at the top. You would see these bags and theshiny things in their holes shift around unpredictably in a waythat perplexes and terries you, although of course you do notrealise that you are perplexed and terried. Gopniks scenariomay not be as imaginary as it seems. Autistic individuals areknown to be decient in knowledge about the mind, andsometimes act as if they viewother people as unpredictableandscary in much this way.

The plan of this article is as follows: The article begins witha history of this research area. Next comes a summary of itsmain theories and research ndings. Thearticle then concludeswith some speculations about the future of the area.

History

As with so many other areas of cognitive development, thehistory of this one mainly begins with Piaget (Flavell&Miller,1998; Shantz, 1983). A central Piagetian claim was thatchildren begin development by being cognitively egocentric(Flavell, 1992). By this, Piaget meant that they initially do notknow that there are such things as conceptual, perceptual, andemotional perspectives or points of view. As a result, theynaturally cannot be aware that they themselves have suchperspectives vis-a`-visexternal objects and events, or that othersdo, or that their own perspectivemay not be the same as thoseof others, or that they may be unwittingly reporting their ownperspectives when asked to report another persons. Piaget alsoconsidered as egocentric children who have some awarenessthat perspectives exist but who arenot skilled at discriminatingtheir own from another persons. Piaget and his colleaguesused egocentrism and other concepts to interpret their

International Journal of Behavioral Development 2000 The International Society for the2000, 24 (1), 1523 Study of Behavioural Development

Correspondence should be addressed to John H. Flavell, Departmentof Psychology, Stanford University, Stanford, CA94305-2130, USA;e-mail: avell@psych.stanford.edu.

16 FLAVELL / MENTAL WORLD

developmental studies of a wide variety of social-cognitivetopics: Perceptual perspective-taking; egocentric communica-tion; the misattribution of mental characteristics to physicalobjects (animism) and physical characteristics tomental events(realism); and understanding of thoughts, dreams, intentions,and morality. Research on these and related topics stillcontinues, although usually not from a Piagetian theoreticalperspective (e.g., Feineld, Lee, Flavell, Green, & Flavell, inpress; Flavell, 1992; Flavell, Green, &Flavell, 1995b; Woolley&Wellman, 1992). Shantz (1983) also describes more recentstage theories of various aspects of social-cognitive develop-ment in the Piagetian tradition byDamon, Selman, and Turiel.She also summarises numerous studies of perspective-takingand related Piagetian topics by Borke, Chandler, Feffer,Flavell, Selman, and many other researchers. There is wide-spread agreement today that young children are not as totallyegocentric as Piaget believed them to be, but also thatperspective-taking abilities and related psychological knowl-edge do showmarked increases with age, much as he said theydid. Those of us trying to peer into the ontogenesis ofknowledge about the mind are clearly standing on Piagetsshoulders.

A second wave of theory and research in this general areawas the extensive work on metacognitive development thatbegan in the early 1970s. Useful reviews of this large literatureinclude Brown, Bransford, Ferrara, and Campione (1983),Flavell, Miller, and Miller (1993), Kuhn (1999), Moshman(1998), and Schneider and Bjorklund (1998). Metacognition(cognition about cognitionhence the meta) has beendened as any knowledge or cognitive activity that takes asits cognitive object, or that regulates, any aspect of anycognitive activity (Flavell et al., 1993, p. 150). It is a broadconcept that encompasses peoples knowledge about thenature of people as cognisers, about the nature of differentcognitive tasks, and about possible strategies for coping withdifferent tasks. It also includes executive skills for monitoringand regulating ones own cognitive activities. The majority ofdevelopmental studies classied as metacognitive have inves-tigated childrens metamemory, that is, their knowledge aboutvariables affecting memory performance and, especially, theirknowledge and use of memory strategies. The term has alsobeen applied to numerous studies of childrens cognitionconcerning comprehension, communication, language, per-ception, and attention, and problem solving. Research in themetacognitive development tradition is still being done,although it is not the hot topic it used to be.

The third wave in our history is still very much in motion,and is the primary concern of this article. It is commonlyreferred to as theory-of-mind research. Prior to about 1983,most investigators of childrens knowledge about the mentalworld would probably classify their work as either metacogni-tive or in the general Piagetian tradition. Today, most wouldsay they are doing one or another kind of theory-of-mindresearch. In fact, they would likely use that label as ashorthand, easily recognisable characterisation of the generalline of work they are in even if they were not convinced, assome are not, that children actually acquire bona de theoriesof mind rather than just knowledge and skills concerning it.Since the mid 1980s, theory-of-mind research has been one ofthe liveliest, most productive research areas in all of develop-mental psychology. I predict that it will continue to be so forsome time to come. How did this third wave come to be?

In the 1978 issue of Behavioral andBrain Sciences, Premack

and Woodruff (p. 515) reported some research in which theyattempted to test whether chimpanzees have a theory of mind,which they dened as follows:

An individual has a theory ofmind if he imputesmental states tohimself and others. A system of inferences of this kind isproperly viewed as a theory because such states are not directlyobservable, and the system can be used to make predictionsabout the behavior of others. As to the mental states thechimpanzees may infer, consider those inferred by our ownspecies, for example, purpose or intention, as well as knowledge,belief, thinking, doubt, guessing, pretending, liking, and so forth.

In their commentaries on this article, three philosophersindependently suggested that one might be able to nd outwhether an animal possessed the concept of belief in somethinglike the following fashion (Bennett, 1978; Dennett, 1978;Harman, 1978). The subject animal sees another individualput object X in container A and then leave the scene. Thesubject then sees someone elsemoveX from container Aintocontainer B while the individual is still absent. The subjectanimal should then be credited with some understanding ofbelief if it acts as if it expects that the returning individual willsearch for X in A rather than B: If the subject chimpanzeeexpects the second chimpanzee to reach into the pot whichoriginally contained the banana, that would seem to show thatit has a conception of mere belief (Dennett, 1978, p. 557).Philosophers and developmental psychologists consider false-belief tasks to be better tests of the concept of belief than aretrue-belief tasks because children could be correct on true-belief tasks by egocentrically assuming that others know whatthey themselves know and just reporting the true state ofaffairs.

These ideas were taken up in the early 1980s by twoAustrian psychologists, Josef Perner and Heinz Wimmer. In apioneering and highly inuential series of studies, they used theunexpected transfer method proposed by the philosophersto test young childrens understanding of false belief (Wimmer& Perner, 1983). Similarly, Bretherton and her colleaguesexamined infants gestural and verbal communication forevidence that infants have what Premack and Woodruff(1978)havecalled a theory ofmind (Bretherton, McNew, &Beeghly-Smith, 1981, p. 339). Around the same time, Well-man and his co-workers had independently begun to con-ceptualise childrens developing metacognitive knowledge andunderstanding of mental terms as the development of a theoryof mind (e.g., Johnson &Wellman, 1980; Shatz, Wellman, &Silber, 1983; Wellman, 1983, 1985). In addition, a number ofother researchers who had not yet begun to conceptualisechildrens social-cognitive development in quite this way hadbeen doing research that subsequently became part of thetheory-of-mind movement. An example would be the work onchildrens knowledge about perception and about the appear-ance-reality distinction by Flavell and colleagues (e.g., Flavell,Flavell, & Green, 1983; Flavell, Flavell, Green, & Wilcox,1980; see Astington, Harris, & Olsen, 1988, for other suchprojects).

The movement was given added identity and coherence bytwo conferences that were held in the spring of 1986: theInternational Conference on Developing Theories of Mind,organised by Janet Astington, Lynd Forguson, Alison Gopnik,and David Olson at the University of Toronto, and theWorkshop on Childrens Early Concept of Mind, organised by

Paul Harris at the University of Oxford. The presentationsgiven at these two conferences were later published in a bookentitled Developing Theories of Mind (Astington et al., 1988),and themovement was ofcially launched. Alook through thismilestone publication provides an immediate sense of thebroad and diverse array of acquisitions judged to be instancesof theory-of-mind development, and more have been addedsince.What it does not convey is thehigh excitement felt by theconference participants as they sensed the birth of a newapproach to the development of childrens knowledge aboutthe mental world.

The rest, as they say, is history. Publications concerned withtheory-of-mind development must now number well up in thehundreds and the ow continues unabated. Reviews of workon this topic include Astington (1993), Bartsch and Wellman(1995), Carpenter, Nagell, and Tomasello (1998), Flavell(1999), Flavell and Miller (1998), Hala (1997), Mitchell(1997), Moore (1996), Taylor (1996), Wellman (1993), andWellman and Gelman (1998).

It may be helpful in understanding the metacognition andtheory-of-mind approaches to compare and contrast them. In ageneral sense, researchers in both traditions share the sameoverall objective, that is, to investigate the development ofchildrens knowledge and cognition about mental phenomena.In fact, most psychologists would probably consider the termsmetacognition and theory of mind as being more or lesssynonymousas just alternative ways of designating the samegeneral set of cognitive phenomena. This sense of synonymityis further heightened by the extensive use of the expressionmetarepresentational in the theory-of-mind developmentliterature: Metarepresentational and metacognitive soundlike they mean pretty much the same thing.

Despite these commonalities, the research literatures inthese two areas have been surprisingly distinct and uncon-nected. Most theory-of-mind articles donot cite research in themetacognitive development traditionfor example, researchon metamemory developmentand conversely, most meta-cognitive development articles do not refer to work in thetheory-in-mind tradition (for a clear exception, see a recentchapter by Kuhn, 1999). One gets a further sense ofdiscontinuity when one looks for the adult counterparts ofthese two research traditions. There is currently a fair amountof psychological research being done with adults under therubric metacognitionon feelings of knowing, for example(Jost, Kruglanski, & Nelson, 1998; Metcalfe & Shimamura,1994)but very little that is construed as research on adulttheory of mind.1

Why this lack of connection? I do not think it is due toprovincialism or lack of vision on the part of researchersconcerned. More likely, it is because the two traditions havetended to focus on different developments within the broadcognition-about-cognition umbrella.

Most theory-of-mind studies have investigated childrensinitial knowledge about our most basic mental statesdesires,percepts, beliefs, knowledge, thoughts, intentions, feelings, andso on. Researchers in this tradition attempt to determine whatchildren of different ages know about the existence andbehaviour of these various states, and also what they knowabout how mental states are causally linked to perceptual

inputs, to behaviour, and to other mental states. For example,do young children understand what it means to know some-thing, or do they realise that unsatised desires typically causenegative feelings and renewed behavioural efforts to satisfythese desires?

In contrast, students of metacognitive development haveusually focused more on task-related mental activitiesoften,on what one should dowith ones mind in trying to solve someproblem or task. These metacognitive activities includestrategies for making cognitive progress on various tasks andproblemson memory or comprehension tasks, for exampleand also attempts to monitor that progress. Much of themetacognition studied is therefore problem-centred and goal-oriented; one could think of it as a kind of applied theory-of-mind.

Becausemost theory-of-mind researchers havebeen lookingfor the origins and earliest expressions of knowledge about themost elementary types of mental states (desires, beliefs, etc.),they have tended to study infants and young childrenpredominantly. Conversely, because the knowledge and skillsmetacognition researchers investigate usually presuppose theprior acquisition of some understanding of these states, theyhavemainly tested older children and adolescents. One cannottest for a childs understanding of memory strategies (meta-cognition) if the child is too young even to know whatremembering something means (theory-of-mind).

Because it has this applied theory-of-mind focus, mostmetacognitive development research is concerned with whatthe subject knows about how to use his/her own mind ratherthan somebody elses. How often other people or people ingeneral use their minds in task situations may be useful asmodels for how the subject should use his/hers, but it is thesubjects own use or non-use that is usuallyof primary interest.In contrast, it is the subjects understanding of some otherpersons mind, or of minds in general, that is usually ofconcern in theory-of-mind studies. For example, in a false-belief task the child subject is typically asked what a naiveotherchild would think is in the cookies boxthe deceptive boxwhich the subject has just learned really contains crayonsrather than cookies.

Main theories and ndings

Figure 1 illustrates the main directions that theory-of-minddevelopment research has taken since it began in the early1980s. Much of the earliest workwas focused on documentinga striking improvement between 3 and 5 years of age inchildrens performance on various false-belief (FB), appear-ance-reality (AR), and Level 2 visual perspective-taking (PT)tasks. Thus, for example, older but not younger preschoolerswere usually found to show an understanding that the naiveother child cited in the rst paragraph of this article wouldfalsely believe that the cookie box contains cookies (falsebelief), that a fake rock looks like a rock but is really a sponge(appearance-reality), and that a picture bookoriented correctlyfor them on the tablewill look upside down to a person seatedopposite (Level 2 visual perspective-taking).

From those beginnings work has progressed more or lessconcurrently in a variety of directions, as shown by the arrowsin Figure 1. Researchers have charted the development ofchildrens understanding of many additional mental states.They have elaborated several classes of theories to explain

INTERNATIONAL JOURNAL OF BEHAVIORAL DEVELOPMENT, 2000, 24 (1), 1523 17

1 There is relevant work here in the area of adult social cognition, to be sure,but it is scarcely ever presented as research on adult theory of mind (i.e., asresearch on the adult outcome of a process of theory-of-mind development).

18 FLAVELL / MENTAL WORLD

theory-of-mind development. They have extended theirinquiries both downward into infancy and upward into middlechildhood and adolescence. Afew attempts havebeen made toidentify child rearingor other variables that predict facilitywiththeory-of-mind tasks (antecedents) and to identify behavioursthat such facility might mediate (consequents). Finally,investigators have examined differences in theory-of-mindunderstandingsdifferences within the same culture, differ-ences between cultures, and differences between humans andother primates. These points are briey elaborated in thesections that follow. Fuller discussion of them can be found inthe substantive reviews cited earlier.

TheoriesSeveral types of theories have been offered as explanations forthe development of childrens mentalistic understanding. Oneis the so-called theory theory (Gopnik & Meltzoff, 1997;Gopnik&Wellman, 1994; Perner, 1991; Wellman&Gelman,1998). Theory theorists argue that our knowledge about themind comprises not a formal scientic theory but an informal,everyday framework or foundational theory. Anumber ofsteps in childrens progression toward the adult theory of mindhave been described. For instance, Bartsch and Wellman(1995) have argued that children begin with a desire psychology,then progress to a desire-belief psychology, and nally attain ouradult belief-desire psychology, in which one recognises that whatpeople believe, as well as what they desire, cruciallyaffects how

they behave. Theory theorists argue that experience plays amajor formative role in childrens theory-of-mind develop-ment.

In contrast, modularity theorists such as Leslie(1994)arguethat young children are not acquiring a theory about mentalrepresentations at all. Rather, Leslie postulates the acquisitionthrough neurological maturation of a succession of threedomain-specic and modular mechanisms for dealing withagents versus nonagent objects. Although experience may benecessary to trigger the operation of these mechanisms, it doesnot determine their nature. Baron-Cohen (1995) also has adevelopmental theory involving innate or early-maturingmodular mechanisms dedicated to mental state computations.

Harris (1992) and others have proposed yet a thirdapproach. According to their simulation theory, childrenbecome able to compute the mental states of other peoplethrough a kind of role-taking or simulation process. Whatdevelops is the ability to make increasingly accurate simula-tions of this kind. Like theory theorists, simulation theoristsalso assume that experience plays a crucial formative role, inthat it is through practice in role taking that children improvetheir simulation abilities.

Still other developmentalists believe that young childrensfailures on false-belief and other theory-of-mind tasks may becaused by more domain-general limitations in executivefunctioning (e.g., Carlson, Moses, & Hix, 1998; Hughes,1998). For example, an inability to inhibit a dominant, ready-to-go response could cause the child subject to blurt out the

Figure 1. Overview of research directions in the area of theory-of-mind development.

cognitively salient real contents of the cookies box when askedwhat the naive other child thought it contained. Otherinvestigators have argued that the tasks may bemisunderstoodby young children ormaynot be engagingenough to elicit theirbest thinking (Flavell &Miller, 1998).

How shall we evaluate all these different theories? It seemslikely to me that an adequate theory here will nally have toinclude elements from all of them:

That is, the following seem likely: (a) that development in thisarea builds from innate or early maturing people-readingcapacities; (b) that we have some introspective ability that wecan and do exploit when trying to infer themental states of othercreatures who are like ourselves but in a different psychologicalsituation (e.g. ignorant of the facts, differently motivated);(c) that much of our knowledge of the mind can becharacterized as an informal theory; (d) that improvedinformation-processing and other abilities (e.g. linguistic skills)enable and facilitate theory of mind development (and certainlyhelp children show what they know on theory-of-mind tasks);and (e) that a variety of experiences serve to engender andchange childrens conceptions of the mental world and theirability to use these conceptions in predicting and explainingtheir own and other peoples behaviour. (Flavell, 1999, p. 27).

Developments during infancyInfants areborn with or develop earlya number of abilities anddispositions that will help them learn about people. They ndhuman faces, voices, and movements highly interesting. Theyhave impressive abilities to perceptually analyse and discrimi-nate human stimuli. They seem impelled to attend to andinteract with other people and they certainly impel otherpeople to attend to and interact with them. Babies responddifferently to people than they do to objects and seem to expectpeople to behave differently than objects do (Poulin-Dubois,1999). More specically, we could say that they come toconstrue people as compliant agents, that is, as objects thatare self-propelled and capable of independent movement(agents) but also inuenceable at a distance by communicativesignals (compliant). One can hardly imagine a more felicitousinitial design for a creature destined for theory-of-minddevelopment.

Late in the rst year infants start to learn that peoplesbehaviour possesses aboutness or intentionality (inten-tionality in a broad sensenot just the narrow sense of onpurpose). An individual behaviour is about an object in thissense if the individual perceptually attends to it, labels it, thinksabout it, wants it, fears it, intends or tries to get it, or relates toit in any other psychological way. Infants do a variety of thingsthat reect a beginning awareness of intentionality. They try toengender new aboutness in others through various commu-nicative gestures, such as pointing to or vocalising about anobject and checking to see if the other person attends to it.They also develop skill at reading the aboutnesses the otherperson already has going, as when they follow the personsgaze. Carpenter et al. (1998) have recently documented athree-step developmental sequence in which infants progressfrom sharing to following to directing others attention andbehaviour. Studies byMeltzoff (1995) have also demonstratedthat 18-month-olds can infer what action another person istrying to perform (e.g., attempting to pull one object awayfrom another object to which it is attached), even though theperson is unsuccessful in the attempt (does not succeed in

pulling it away) and therefore never actually demonstrates theintended action. This suggests that infants of this age havesome sense that peoples actions are intentional and goal-directed. Precursors of such understanding can even beobserved in early infancy (Woodward, 1998). By age 18months, infants understand that they should give an experi-menter a food that she reacts to with pleasure rather than onetowards which she acts disgusted, even when they themselvesprefer the latter food (Repacholi & Gopnik, 1997); thissuggests at least some limited ability to reason nonegocen-trically about peoples desires.

Infants also recognise that it is the adults attentional focusrather than their own that gives clues as to the adultsreferential intent when the adult labels an object (Woodward&Markman, 1998). Similarly, they develop the ability to learnwhat an object is like by reading the adults attentional focuswhen the adult is expressing a positive or negative emotionalreaction to it (a process called social referencing). For instance,theymay selectively avoid an object towards which their parentshows negative affect. Thus, by 12 months or so they canrecognise that the adults emotional display refers to or isabout a particular objectmuch as they can recognise that theadults spoken label refers to or is about a particular object(Moses, Baldwin, Rosicky, &Tidball, in press). By the end ofinfancy children may also do other things suggestive of abeginning understanding of human psychology, such as tryingto comfort people in distress and correctly using mental stateterms such as want and see.

Later developmentsA very large literature has accrued since the early 1980s ontheory-of-mind acquisitions that occur subsequently to theinfancy period. What follows is a brief synopsis of some of themajor ndings, organised by type of mental state.

Visual perception. During the early preschool period childrenalreadyunderstand that a personwill seean object if and only ifthe persons eyes are aimed in the general direction of theobject, and if there are no vision-blocking obstacles interposedbetween the person and the object (Flavell, 1992). With thisunderstanding, they areable to do simple, nonegocentric visualperspective-taking; for example, they can infer that you maysee something that they do not and vice versa (referred to asLevel 1 knowledge about visual perception). Later in thepreschool period they go on to recognise that the same thingmay present different visual appearances to two people if theyview it from different positions (called Level 2 knowledge).

Attention. As alreadymentioned, even infants pay attention toother peoples attending and seem to have someunderstandingof its implications. In subsequent years they come to appreciatethat attention is selective and limited, and that different peoplemay mentally represent the same attended-to input differently(Fabricius &Schwanenugel, 1994; Flavell, Green, &Flavell,1995a; Pillow, 1995).

Desires. By the age of 3 years, children are not only usingsome desire terms correctly, they also seem to grasp simplecausal relations among desires, outcomes, emotions, andactionssuggestive evidence that they are developing some-thing like an implicit theory. For example, they understandthat people will feel good if they get what they want and feel

INTERNATIONAL JOURNAL OF BEHAVIORAL DEVELOPMENT, 2000, 24 (1), 1523 19

20 FLAVELL / MENTAL WORLD

bad if they do not, and that people will quit searching if theynd a sought-after desired object but will keep searching if theydo not (Bartsch &Wellman, 1995).

Emotions. Although we do not know yet whether infantsactually impute inner feelings to people who display emotions,it seems clear that young preschoolers evidence an under-standing of emotions as experiential states of persons, asdistinguished from the actions (e.g., hitting) and expressions(e.g., smiling) that emotions cause, and they distinguishbetween the subjective emotional experiences of differentindividuals (Wellman, Harris, Banerjee, & Sinclair, 1995,p. 118). In later years, children gradually learnmore advancedtruths about emotions, for example, that people do not alwaysreally feel what they appear to feel, that their affective reactionsto an event may be inuenced by earlier emotional experienceswith similar events or by their current mood, and that peoplecan experience two conicting emotions more or less simulta-neously (Flavell &Miller, 1998).

Beliefs and related mental representations. There have been agreat many studies of childrens developing understanding ofserious mental representations, that is, nonpretence mentalstates such as beliefs that are meant to represent realityaccurately (Flavell &Miller, 1998). Themajority of these havedealt with childrens comprehension of representations thatdiffer from person to person or differ from reality: Theappearance-reality distinction, Level 2 knowledge of visualperception, interpretation, and constructive processing, decep-tion, and most studied of all, false belief. The distinctionbetween perceptual appearance and reality is conceptuallysimilar both to the distinction between false belief and realityand to theLevel 2 distinction between two different perceptualperspectives. Consistent with this fact, there is some correla-tional evidence suggesting that these three distinctions tend todevelop synchronously, within subjects, during the preschoolyears. As noted in the Theories section, however, exactly whatfalse-belief and appearance-reality tasks measure remainsuncertain; someresearchers (includingme) believetheymainlyassess the childs nascent understanding of mental representa-tion, but others disagree. The evidence indicates thatchildrens knowledge about mental representations continuesto increase after the preschool period. In particular, it is notuntil middle childhood and later that children appear to gainany substantial understanding of the mind as an active,interpretive, constructive processor (e.g., Carpendale &Chandler, 1996). For instance, understanding that peoplesinterpretation of an ambiguous event may be inuenced bytheir preexisting biases or expectations seems to be a middlechildhood insight (Pillow &Henrichon, 1996).

Knowledge. Young preschoolers appear to be unclear aboutjust what it means for someone to know something and abouthow knowledge is acquired (Flavell &Miller, 1998). Even 4-and 5-year-olds may claim that they have always knowninformation that theyhave just learnedduring theexperimentalsession (Taylor, Esbensen, & Bennett, 1994). An importantearly-middle childhood discovery is that perceptual informa-tion has to be adequate as well as merely present to engenderknowledge. For example, children cometo realise that apersonoften cannot be certain of an objects identity when only a littlebit of it is visible; this realisation is another example of theirburgeoning conception of the mind as an interpretive device.

Pretence.. The acquisition of pretend-play skills during earlychildhood is currently viewed as part of the development ofchildrens knowledge about the mind, thanks largely to animportant analysis by Leslie (1987, 1994). Leslie argues thattheability to understand pretence and the ability to understandfalsebelief and other mental states aremediated by a common,early-maturing theory-of-mind module. This argument hassome plausibility: Pretending that and believing that areboth prepositional attitudes. Moreover, adults regard both asmental representations or construals of something as being acertain wayeither for real (belief) or just temporarily, for playpurposes (pretence). Nevertheless, Leslies claim is currentlycontroversial (Lillard, 1998a). The related topic of childrensunderstanding of imagination is also receiving considerablestudy (Woolley, 1995).

Thinking. Children achieve some important elementaryknowledge and skills concerning thinking during the earlypreschool years. For example, they come to construe it as aninternal human activity that refers to or represents real orimaginary things. However, there are also important knowl-edge and skills concerning thinking that preschoolers clearlylack (Flavell et al., 1995b; Flavell &ODonnell, 1999). Theyare not aware that people are continually experiencing mentalcontent spontaneously in an ever-owing stream of conscious-ness. For example, unlike older children, preschoolers do notconsistently attribute anymental activity at all to a person whojust sits quietly, waiting. These same difculties are equallyevident when preschoolers areasked to report their ownmentalactivity rather than another persons (Flavell, Green, &Flavell,2000). That is, they tend to be verypoor at reporting their ownrecent or present thinking, even in situations especiallydesigned to make such introspection extremely easy (but seeEstes, 1998 for an exception). They also do not differentiatevery clearly between the mental contents of conscious andunconscious states (Flavell, Green, Flavell, & Lin, 1999). Inparticular, just as they tend to attribute too little ongoingideation to a conscious person (they are unaware of the streamof consciousness), they also attribute too much to anunconscious one (they attribute conscious thought and self-awareness to people who are unconscious).

Differences in developmentIntracultural differences. Investigators have recently been ex-amining three kinds of differences in development: Intracul-tural, intercultural, and interspecies (Flavell &Miller, 1998).Regarding intracultural differences, researchers have identiedsome social experiences that appearto facilitate theory-of-minddevelopment (Bartsch & Estes, 1996). For example, Jenkinsand Astington (1996) and Perner, Ruffman, and Leekam(1994)have shown that young childrenwho havemore siblingsto interact with perform better on false-belief tasks than thosewho have fewer or none. Likewise, deaf children whose hearingparents are not uent in sign language (as most are not)performmore poorly on a false-belief test than deaf children ofuent-signing deaf parents (e.g., Peterson & Siegal, 1997).Such ndings suggest the importance of social-communicativeexperiences for the development of mentalistic understanding.This understanding in its turn undoubtedly facilitates thedevelopment of social skills (Watson, Nixon, Wilson, &Capage, 1999). The most striking intracultural differences,however, are manifest in the severe decits in theory-of-mind

development exhibited by autistic children and adults (Baron-Cohen, Tager-Flusberg, & Cohen, 1993). How well thesetragic decits can be reduced or compensated for by training iscurrently under study (Wellman, Baron-Cohen, Gomez,Swettenham, &Toye, manuscript in preparation).

There are also signicant intracultural differences amongunimpaired individuals. Dweck and her co-workers havedocumented important individual differences in peoplesimplicit theories about intelligenceand other human attributes(Dweck, Chiu, & Hong, 1995). Textbooks in the elds ofpersonality, social psychology, and social cognition alsodescribe many other ways that normal adults differ from oneanother in their naive theories and knowledge regardingthemselves and other people; great works of literature are aneven richer source. And, of course, psychologists and otherscientists haveespoused widely different conceptions of humancognition and personality over the years: Just think of thedifferences between B.F. Skinners and Freuds views of themind.

Intercultural differences. The question of between-culturesimilarities and differences in theory-of-mind development isa fascinating one. How universal are the developmentsdescribed in this article? An important review of the existingevidencemostly from ethnographic studiessuggests thatthere are important differences between cultures in adulttheories of mind (Lillard, 1998b). However, there is alsoreason to think that there are some deep universals as well(Avis &Harris, 1991; Wellman, 1998).

Interspecies differences. Researchers havealso tested for theory-of-mind knowledge and abilities in other primates. Recentexperimental workwith chimps suggests that theymay actuallybe less knowledgeable in this area than originally thought (Call& Tomasello, 1999; Reaux, Theall, & Povinelli, 1999). Forinstance, Reaux et al. (1999) present evidence suggesting thatchimps may possess a behaviouristic rather than mentalisticconception of seeing. That is, although they follow a personsgaze, they appear not to understand that the person sees andknows about things as a result of directing his/her gazeat them.

Future Research

Researchers have learned a considerable amount about thedevelopment of childrens mentalistic understanding, espe-cially from the theory-of-mind investigations of the past twodecades. What will the next few decades bring?The followingare some guesses (see also Flavell & Miller, 1998, pp. 882887).

As indicated in the Theories section, there is no shortage oftheories intended to explain how children acquire an under-standing of the mental world. It is a safe prediction that thefuture will see further theoretical and experimental work in thisarea. In particular, how the development of executivefunctioning may be related to theory-of-mind developmentwill likely continue to be the subject of considerable enquiry(Hughes, 1998).

Infant development is the hottest research area in thetheory-of-mind eld currently and will probably continue to befor some time to come. It seems unlikely that researchers willdiscover new theory-of-mind acquisitions in this age period(i.e., things we did not know developed). Rather, they will be

better able to date, describe, and explain developments alreadyunder study. The nonverbal research measures presently usedare ingenious (e.g., looking time) but one can hope for evenbetter ones in the future. Nonverbal measures will alsocontinue to be used to compare the performance of otherprimates with that of human children. The recent workof Calland Tomasello (1999) and Reaux et al. (1999) demonstratehow fruitful this research strategy can be. My bet is that suchcomparisons will continue to show a lack of signicantunderstanding of mental states in nonhuman subjects.

Toward the other end of ontogenesis, we can expect to seemore research on middle childhood, adolescent, and adultunderstanding. What do adolescents and adults know orbelieve about themental world that elementary schoolchildrendo not, and what understanding do the latter have thatpreschoolers do not? We still lack adequate answers to thesequestions. My guess is that there exist advanced competenciesin this area that we have not yet identied. How similar thesemore complex and subtle forms of understanding are fromadult to adult, within and between cultures, is an importantand clearly researchable question. One possible differencebetween children and adults that has not been explored muchmay lie, not in what mental states they are capable of havingconscious thoughts about, but rather in how easily and howoften they are spontaneously aware of their own or otherpeoples mentation. It is one thing to know what thoughts andfeelings are and that people have them, but quite another tospontaneously detect or infer their occurrence when theyhappen. Such increased sensitivitymay be one of the productsof lifes advanced courses on mind-reading.

In future research investigators will also try to tell longer andricher developmental stories about each acquisition they study.Consider, for example, the acquisition of an understanding ofbelief (Flavell & Miller, 1998, pp. 873875). Even if oneaccepts (still controversial) that 3-year-olds do not understandbeliefs but 4-year-olds do, it is becoming clear that thisdevelopmental story is incomplete. For the later part of thestory, there is evidence that children continue to acquireknowledge about beliefs after the ageof 4 years (Carpendale&Chandler, 1996; Fabricius & Imbens-Bailey, 2000). As to theearly part, Clements and Perner (1994) have found that young3-year-olds who respond incorrectly to standard false-belieftask questions nonetheless show by their eye movements thatthey may have some sort of rudimentary, implicit under-standing of false beliefs. In a recent review, Haith and Bensoncalled for a graded approach to understanding infantcognition, one that considers the succession of partialaccomplishments that is likely to occur in each domain(1998, p. 245). Future research on theory-of-mind develop-ment will likely follow their prescription and try to identify anumber of different levels of understanding of each mentalstate concept.

There is very little research to date on how new acquisitionsin this area translate into changes in childrens everyday socialand cognitive behaviours (Flavell & Miller, 1998). We willlikely see much more of this kind of research in the future.There will also be efforts to nd out how best to help childrenwho need such help to acquire socially and academicallyusefultheory-of-mind competencies.

Whatever else it may turn out to be, the 21st century seemscertain to be the Century of the Brain. My nal prediction(more of a hope than a prediction) is that advances inneuroscience will help us better understand the childs

INTERNATIONAL JOURNAL OF BEHAVIORAL DEVELOPMENT, 2000, 24 (1), 1523 21

22 FLAVELL / MENTAL WORLD

developing understanding of the mental world (Stone, Baron-Cohen, &Knight, 1998).

Manuscript received September 1999

References

Astington, J.W. (1993). The childs discovery of the mind. Cambridge, MA:Harvard University Press.

Astington, J.W., Harris, P.L., & Olson, D. (Eds.) (1988). Developing theories ofmind. Cambridge, UK: Cambridge University Press.

Avis, J., & Harris, P.L. (1991). Belief-desire reasoning among Baka children:Evidence for a universal conception of mind. ChildDevelopment, 62, 460467.

Baron-Cohen, S. (1995). Mindblindness: An essay on autism and theory of mind.Cambridge, MA: MIT Press.

Baron-Cohen, S., Tager-Flusberg, H., & Cohen, D.J. (1993). Understandingother minds: Perspectives from autism. Oxford, UK: Oxford University Press.

Bartsch, K., &Estes, D. (1996). Individual differences in childrens developingtheory of mind and implications for metacognition. Learning and IndividualDifferences, 8, 281304.

Bartsch, K., &Wellman, H.M. (1995). Children talk about the mind. New York:Oxford University Press.

Bennett, J. (1978). Some remarks about concepts. Behavioral and Brain Sciences,1, 557560.

Bretherton, I., McNew, S., & Beeghly-Smith, M. (1981). Early personknowledge as expressed in gestural and verbal communication: When doinfants acquire a theory of mind? In M. Lamb &L. Sherrod (Eds.), Socialcognition in infancy (pp. 333373). Hillsdale, NJ: Erlbaum.

Brown, A.L., Bransford, J.D., Ferrara, R.A., & Campione, J.C. (1983).Learning, remembering, and understanding. In P.H. Mussen (Series Ed.),J.H. Flavell & E.M. Markman (Eds.), Handbook of child psychology: Vol. 3,Cognitive development (pp. 77166). New York: Wiley.

Call, J., &Tomasello, M. (1999). Anonverbal false belief task: The performanceof children and great apes. Child Development, 70, 381395.

Carlson, S.M., Moses, L.J., &Hix, H.R. (1998). The role of inhibitory processesin young childrens difculties with deception and false belief. ChildDevelopment, 69, 672691.

Carpendale, J.I., & Chandler, M.J. (1996). On the distinction between falsebelief understanding and subscribing to an interpretive theory of mind. ChildDevelopment, 67, 16861706.

Carpenter, M., Nagel, K., & Tomasello, M. (1998). Social cognition, jointattention, and communative competence from 915 months of age.Monographs of the Society for Research in Child Development, 63, (4, SerialNo. 255).

Clements, W.A., &Perner, J. (1994). Implicit understanding of belief. CognitiveDevelopment, 9, 377395.

Dennett, D.C. (1978). Beliefs about beliefs. Behavioral and Brain Sciences, 1,568570.

Dweck, C.S., Chiu, C.Y., &Hong, Y.Y. (1995). Implicit theories and their rolein judgements and reactions: A world from two perspectives. PsychologicalInquiry, 6, 267285.

Estes, D. (1998). Young childrens awareness of their mental activity: The caseof mental rotation. Child Development, 69, 13451360.

Fabricius, W.V., & Imbens-Bailey, A.L. (2000). False belief about false beliefs.In P. Mitchell &K. Riggs (Eds.), Childrens reasoning and themind. Hove, UK:Psychology Press.

Fabricius, W.V., & Schwanenugel, P.J. (1994). The older childs theory ofmind. In A. Demetriou &A. Efklides (Eds.), Intelligence, mind, and reasoning:Structure and development (pp. 111132). Amsterdam: Elsevier.

Feineld, K.A., Lee, P.P., Flavell, E.R., Green, F.L., &Flavell, J.H. (in press).Young childrens understanding of intention. Cognitive Development.

Flavell, J.H. (1992). Perspectives on perspective taking. In H. Beilin &P. Pufall(Eds.), Piagets theory: Prospects and possibilities (pp. 107139). Hillsdale, NJ:Erlbaum.

Flavell, J.H. (1999). Cognitive development: Childrens knowledge about themind. Annual Review of Psychology, 50, 2145.

Flavell, J.H., Flavell, E.R., & Green, F.L. (1983). Development of theappearance-reality distinction. Cognitive Psychology, 15, 95120.

Flavell, J.H., Flavell, E.R., Green, F.L., & Wilcox, S.A. (1980). Youngchildrens knowledge about visual perception. Effect of observers distancefrom target on perceptual clarity of target. Developmental Psychology, 16, 1012.

Flavell, J.H., Green, F.L., & Flavell, E.R. (1995a). The development ofchildrens knowledge about attentional focus. Developmental Psychology, 31,706712.

Flavell, J.H., Green, F.L., &Flavell, E.R. (1995b). Young childrens knowledgeabout thinking. Monographs of the Society for Research in ChildDevelopment, 60(1, Serial No. 243).

Flavell, J.H., Green, F.L., & Flavell, E.R. (2000). Development of childrensawareness of their own thoughts. Journal of Cognitive Development, 1, 97112.

Flavell, J.H., Green, F.L., Flavell, E.R., & Lin, N.T. (1999). Development ofchildrens knowledge about unconsciousness. Child Development, 70, 396412.

Flavell, J.H., &Miller, P.H. (1998). Social cognition. In D. Kuhn&R.S. Siegler(Eds.), W. Damon (Series Ed.), Handbook of child psychology: Vol. 2. Cognition,perception, and language (5th ed., pp. 851898). New York: Wiley.

Flavell, J.H., Miller, P.H., &Miller, S.A. (1993). Cognitive development (3rd ed.).Englewood Cliffs, NJ: Prentice-Hall.

Flavell, J.H., &ODonnell, A.K. (1999). Le developpement de savoirs intuitifs a`propos des experiences mentales. Enfance, 51, 267276.

Gopnik, A., &Meltzoff, A.N. (1997). Words, thoughts, and theories. Cambridge,MA: MIT Press.

Gopnik, A., &Wellman, H.M. (1994). The theory theory. In L.A. Hirschfeld &S.A. Gelman (Eds.), Mapping the mind: Domain specicity in cognition andculture (pp. 257293). Cambridge, UK: Cambridge University Press.

Haith, M.M., & Benson, J.B. (1998). Infant cognition. In D. Kuhn & R.S.Siegler (Eds.), W. Damon (Series Ed.), Handbook of child psychology: Vol. 2.Cognition, perception, and language (5th ed., pp. 199254). New York: Wiley.

Hala, S. (Ed.) (1997). The development of social cognition. Hove, UK: PsychologyPress.

Harman, G. (1978). Studying the chimpanzees theory of mind. Behavioral andBrain Sciences, 1, 576577.

Harris, P.L. (1992). From simulation to folk psychology: The case fordevelopment. Mind and Language, 7, 120144.

Hughes, C. (1998). Finding your marbles: Does preschoolers strategic behaviorpredict later understanding of mind? Developmental Psychology, 34, 13261339.

Jenkins, J.M., &Astington, J.W. (1996). Cognitive factors and family structureassociated with theory of mind development in young children. DevelopmentalPsychology, 32, 7078.

Johnson, C.N., &Wellman, H.M. (1980). Childrens developing understandingof mental verbs: Remember, know, and guess. Child Development, 51, 10951102.

Jost, J.T., Kruglanski, A.W. & Nelson, T.O. (1998). Social metacognition: Anexpansionist review. Personality and Social Psychology Review, 2, 137154.

Kuhn, D. (1999). Metacognitive development. In C. Tamis Le-Monda (Ed.),Child psychology: A handbook of contemporary issues. Philadelphia, PA:Psychology Press.

Leslie, A.M. (1987). Pretense and representation: The origins of theory ofmind. Psychological Review, 94, 412426.

Leslie, A.M. (1994). ToMM, ToBy, and agency: Core architecture and domainspecicity. In L.A. Hirschfeld & S.A. Gelman (Eds.), Mapping the mind:Domain specicity in cognition and culture (pp. 119148). Cambridge, UK:Cambridge University Press.

Lillard, A.S. (1998a). Wanting to be it: Childrens understanding of intentionsunderlying pretense. Child Development, 69, 979991.

Lillard, A.S. (1998b). Ethnopsychologies: Cultural variations in theories ofmind. Psychological Bulletin, 123, 333.

Meltzoff, A.N. (1995). Understanding the intentions of others: Re-enactment ofintended acts by 18-month-old children. Developmental Psychology, 31, 838850.

Metcalfe, J., &Shimamura, A.P. (Eds.) (1994). Metacognition. Cambridge, MA:MIT Press.

Mitchell, P. (1997). Introduction to theory of mind: Children, autism and apes.London: Arnold.

Moore, C. (1996). Theories of mind in infancy. British Journal of DevelopmentalPsychology, 14, 1940.

Moses, L.J., Baldwin, D.A., Rosicky, J.G., &Tidball, G. (in press). Evidence forreferential understanding in the emotions domain at 12 and 18 months. ChildDevelopment.

Moshman, D. (1998). Cognitive development beyond childhood. In D. Kuhn &R.S. Siegler (Eds.), W. Damon (Series Ed.), Handbook of child psychology: Vol.2: Cognition, perception, and language (5th ed., pp. 947978). New York:Wiley.

Perner, J. (1991). Understanding the representational mind. Cambridge, MA: MITPress.

Perner, J., Ruffman, T., &Leekam, S.R. (1994). Theory of mind is contagious:You catch it from your sibs. Child Development, 65, 12281238.

Peterson, C.C., & Siegal, M. (1997). Psychological, physical, and biologicalthinking in normal, autistic, and deaf children. InH.M. Wellman &K. Inagaki(Eds.), The emergence of core domains of thought: Childrens reasoning aboutphysical psychological, and biological phenomena. (New Directions for ChildDevelopment), (No. 75, pp. 5570). San Francisco, CA: Jossey-Bass.

Pillow, B.H. (1995). Two trends in the development of conceptual perspective-taking: An elaboration of the passive-active hypothesis. International Journal ofBehavioral Development, 18, 649676.

Pillow, B.H., &Henrichon, A.J. (1996). Theres more to the picture than meetsthe eye: Young childrens difculty understanding biased interpretation. ChildDevelopment, 67, 803819.

Poulin-Dubois, D. (1999). Infants distinction between animate and inanimateobjects: The origins of naive psychology. In P. Rochat (Ed.), Early socialcognition (pp. 257280). Mahwah, NJ: Erlbaum.

Premack, D., &Woodruff, G. (1978). Does the chimpanzee have a theory ofmind?Behavioral and Brain Sciences, 1, 515526.

Reaux, J.E., Theall, L.A., &Povinelli, D.J. (1999). A longitudinal investigationof chimpanzees understanding of visual perception. Child Development, 70,275290.

Repacholi, B.M., &Gopnik, A. (1997). Early reasoning about desires: Evidencefrom 14- and 18-month olds. Developmental Psychology, 33, 1221.

Shantz, C.U. (1983). Social cognition. In P.H. Mussen (Series Ed.), J.H. Flavell& E.M. Markman (Eds.), Handbook of child psychology: Vol. 3. Cognitivedevelopment (pp. 495555). New York: Wiley.

Shatz, M., Wellman, H.M., &Silber, S. (1983). The acquisition ofmental verbs:A systematic investigation of rst references to mental state. Cognition, 14,301321.

Schneider, W., & Bjorklund, D.F. (1998). In D. Kuhn & R.S. Siegler (Eds.),W. Damon (Series Ed.), Handbook of child psychology: Vol. 2. Cognition,perception, and language (5th ed., pp. 467521). New York: Wiley.

Stone, V.E., Baron-Cohen, S., & Knight, R.T. (1998). Frontal lobe contribu-tions to theory of mind. Journal of Cognitive Neuroscience, 10, 640656.

Taylor, M. (1996). A theory of mind perspective on social cognitivedevelopment. In R. Gelman & T. Au (Eds.), E.C. Carterette & M.P.Friedman (Gen Eds.), Handbook of perception and cognition: Vol. 13, Perceptualand cognitive development (pp. 283329). New York: Academic Press.

Taylor, M., Esbensen, B.M., &Bennett, R.T. (1994). Childrens understandingof knowledge acquisition: The tendency for children to report theyhave always known what they have just learned. Child Development, 65,15811604.

Watson, A.C., Nixon, C.L., Wilson, A., &Capage, L. (1999). Social interactionskills and theory of mind in young children. Developmental Psychology, 35,386391.

Wellman, H.M. (1983). Metamemory revisited. In M.T.H. Chi (Ed.), Trends inmemory development research (pp. 3151). Basel: Karger.

Wellman, H.M. (1985). The childs theory of mind: The development ofconceptions of cognition. In S.R. Yussen (Ed.), The growth of reection inchildren (pp. 169206). San Diego, CA: Academic Press.

Wellman, H.M. (1993). Early understanding of mind: The normal case. InS. Baron-Cohen, H. Tager-Flusberg, &D. Cohen (Eds.), Understanding otherminds: Perspectives from autism (pp. 1039). Oxford, UK: Oxford UniversityPress.

Wellman, H.M. (1998). Culture, variation, and levels of analysis in our folkpsychologies. Psychological Bulletin, 123, 3336.

Wellman, H.M., Baron-Cohen, S., Gomez, J.C., Swettenham, J., & Toye, E.Using thought-bubbles helps children with autism acquire an alternative fortheory of mind. Manuscript in preparation.

Wellman, H.M., Cross, D., & Watson, J.K. (1999, April). A meta-analysis oftheory of mind development: The truth about false belief. Poster presented at themeeting of the Society for Research in Child Development, Albuquerque,NM.

Wellman, H.M., & Gelman, S.A. (1998). Knowledge acquisition in functionaldomains. In D. Kuhn and R.S. Siegler (Eds.), W. Damon (Series Ed.),Handbook of Child Psychology: Vol. 2. Cognition, perception, and language (5thed., pp. 523573). New York: Wiley.

Wellman, H.M., Harris, P.L., Banerjee, M., & Sinclair, A. (1995). Earlyunderstandings of emotion: Evidence from natural language. Cognition andEmotion, 9, 117149.

Wimmer, H., & Perner, J. (1983). Beliefs about beliefs: Representation andconstraining function of wrong beliefs in young childrens understanding ofdeception. Cognition, 13, 103128.

Woolley, J.D. (1995). The ctional mind: Young childrens understanding ofimagination, pretense, and dreams. Developmental Review, 15, 172211.

Woolley, J.D., & Wellman, H.M. (1992). Childrens conceptions of dreams.Cognitive Development, 7, 365380.

Woodward, A.L. (1998). Infants selectively encode the goal object of an actorsreach. Cognition, 69, 134.

Woodward, A.L., &Markman, E.M. (1998). Early word learning. In D. Kuhn&R.S. Siegler (Eds.), W. Damon (Series Ed.), Handboook of child psychology:Vol. 2. Cognition, perception, and language (5th ed., pp. 371420). New York:Wiley.

INTERNATIONAL JOURNAL OF BEHAVIORAL DEVELOPMENT, 2000, 24 (1), 1523 23