22
1 Semiotic Potential of Multimodal Experiences for Early Years Readers Faculty of Education University of Manitoba Eighth Annual Graduate Student Symposium March 9 and 10, 2007 Beryl Peters

Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

Embed Size (px)

Citation preview

Page 1: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

1

Semiotic Potential of Multimodal Experiences for Early Years Readers

Faculty of Education University of Manitoba

Eighth Annual Graduate Student Symposium

March 9 and 10, 2007

Beryl Peters

Page 2: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

2

Semiotic Potential of Multimodal Experiences for Early Years Readers

Faculty of Education University of Manitoba

Eighth Annual Graduate Student Symposium

March 9 and 10, 2007

Beryl Peters

I began my journey through the Faculty of Education PhD program in Inclusive Special

Education at the University of Manitoba with a fervent belief in the “semiotic potential of

multimodal experiences for early years readers” although at that time, I did not have the

academic vocabulary or understanding to articulate or define those beliefs. I did not know that I

practiced a multimodal, multiliteracy approach to teaching; I had no idea there was a wide body

of research and educational literature to support multimodal teaching and learning, and I had

never heard of the term “multiliteracies.” Questions from instructors and conversations with

faculty professors led me to examine the field of semiotics and the recent literature on social

semiotics and multiliteracy, multimodal approaches to teaching and learning through an

independent study with Dr. Wayne Serebrin that transformed my thinking about learning through

the arts and resulted in a rewriting of my dissertation proposal to incorporate and reflect this new

learning.

Literacy education is undergoing a paradigmatic shift (Bearne, 2003) towards a pedagogy

of multiliteracies (Cope & Kalantzis, 2000) using a variety of semiotic systems and approaches.

Semiotics, or the study of signs and symbols to convey and share meaning, is as important to

education today as it was to Plato circa 360 BCE. The research proposal for my PhD

dissertation relies and builds on the on the age-old semiotic potential of multimodal experiences

Page 3: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

3

and represents a synthesis of my practice and the foundation and theories of multiliteracy

teaching and learning as informed by the field of semiotics.

The meaning and definition of semiotics has been a debatable subject of research

throughout past centuries (Noth, 1990). The ancient Greeks explored signs and sign systems:

Plato through philosophy; Aristotle through linguistics and poetics; and Hippocrates and Galen in

medicine (Sebeok, 1994). Sebeok describes Hippocrates as both the father of medicine and of

semiotics. Semiotics (then spelled semeiotics) was first coined by Hippocrates to describe and

study symptoms for medical diagnosis (Danesi, 2004).

Discussion and interest in signs and symbol systems continued throughout the centuries

and signs played an important role in theology, linguistics, philosophy, and logic (Danesi, 2004).

In 1690, the formal study of signs known as semiotics was proposed by the British philosopher

John Locke in the concluding Book IV of his “Essay Concerning Human Understanding”. In the

conclusion to Book IV, Locke divides the sciences into three typologies and defines the third

branch as the doctrine of signs:

Thirdly, the third branch may be called Semeiotike, or the doctrine of signs…the

business whereof is to consider the nature of signs, the mind makes use of for the

understanding of things, or conveying its knowledge to others. For, since the things the

mind contemplates are none of them, besides itself, present to the understanding, it is

necessary that something else, as a sign or representation of the thing it considers, should

be present to it: and these are ideas. And because the scene of ideas that makes one man's

thoughts cannot be laid open to the immediate view of another, nor laid up anywhere but

in the memory, a no very sure repository: therefore to communicate our thoughts to one

Page 4: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

4

another, as well as record them for our own use, signs of our ideas are also necessary.

Locke, 1995, p. 607-08

Locke espoused a referential relationship between ideas and words; the “this means that

view—was the starting point for British semiotic theory,” dominating “logical and semiotic

thought for a century after” (Monelle, 1992, p. 13). Ferdinand de Saussure, a Swiss linguist, and

Charles Pierce, an American philosopher, continued pioneering efforts to lay the foundation for

modern semiotic study.

As a linguist, de Saussure (1857-1913) focused his studies on linguistic signs. Agawu

(1991) notes the basic semiotic term of “sign” as defined by de Saussure is a binary entity,

“consisting of the union of the signifier and the signified” (p. 16). The signifier is the actual

phenomenon or sound-image (Saussure, 1992) and the signified is “the concept embodied in a

particular signifier” (Agawu, 1991, p. 16). The linguistic sign is described by de Saussure (1992)

as a “two-sided psychological entity” (p. 37) that goes beyond merely naming a thing to uniting a

concept and a sensory, material sound-image. Solomon (1988) offers an example to illustrate de

Saussure’s concept of signified and signifier. “For Saussure, the ‘signified’ (or meaning) referred

to by the ‘signifier’ dog, for instance, is not a flesh-and-blood animal but a concept that can be

distinguished from our concepts of, say, foxes, wolves, and even cats” (p. 14-15).

De Saussure outlines basic principles of the study he terms semiology. The bond between

the signifier and the signified is arbitrary, meaning not that the speaker can change an established

linguistic sign, but that “it is unmotivated, i.e. arbitrary in that it actually has no natural

connection with the signified” (Saussure, 1992, p. 39). A second, fundamental principle outlined

by de Saussure concerns the linear nature of the signifier; the elements of the auditory, linguistic

signifiers are presented in succession, unfolding in a measurable, linear span of time.

Page 5: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

5

Pierce (1839-1914) has been described as the founder of modern semiotics (Solomon,

1988). Innis (1985) observes that the particular significance of the work of Charles Pierce lies in

the “fundamental trichotomy of the ways a sign can be related via an ‘interpretant’, to its object

and what this threefold relationship tells us about…the process of the production and

interpretation of signs” (Innis, p. vii). Sebeok (1994) declares that Pierce “uniquely reinvigorated

semiotics” (p. 5); “the irreducibly triadic relationship among a sign, its object and its

interpretant…have far-resounding philosophical overtones” (p. 5). Pierce states that the sign

serves as a mediator between the interpreting thought and its object (Hardwick, 1977). Sebeok

(1994) observes that semiotics, as conceived by Pierce, is thus an integrated science of

communication (the exchange of messages) that focuses on the study of signification, or the

underlying system of signs, and the meaning established through signification.

Harste, Woodward, and Burke (1984) exemplify Pierce’s semiotic triad with a

preschooler’s response to a Crest toothpaste box. When shown the box, the child responds

“brush teeth”. According to Pierce’s model, the print and graphics on the box are the sign

standing for something (the object or Crest toothpaste), and indicate a meaning derived or

interpreted from the sign (brush teeth).

Sebeok defines the semiological notions of de Saussaure as a “minor tradition in semiotics

and believes the studies of Locke and Pierce constitute the “major tradition” in the history of

semiotics (Petrilli & Ponzio, 2001). Reviewers of Sebeok’s writings offer caveats to this

perspective, suggesting that the notion of “Pierce as master” is questionable and should be

balanced by an understanding of the contributions of de Saussure and others (Packwood, 1996).

The diverse semiotic traditions including those of Pierce and de Saussure have resulted in varied

and profuse semiotic typologies including communication semiotics, signification semiotics,

Page 6: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

6

referential semiotics, nonreferential semiotics, and interpretation semiotics, to name but a few

(Petrilli & Ponzio, 1998). Noth (1990) notes the various meanings and schools of semiotics and

references sixteen different definitions for semiotics specified by Pelc.

Despite the varying perspectives expressed in the literature and the references to semiotic

jargon (Maeder, Fisher, & Herlofsky, 2003), there are elements common to the diverse

typologies. Eco states, “Semiotics is concerned with everything that can be taken as a sign. A

sign is everything that can be taken as significantly substituting for something else” (1976, p. 7).

Danesi illustrates this semiotic relationship with a simple formula: “Describing and investigating

the nature of the x=y relation constitutes, tout court, the subject matter of semiotics” (2004, p. 4).

Using Danesi’s model of x=y, the meaning of green has many possibilities depending on the

nature of the x/y relationship, the context in which it is located, and who sees and interprets it.

On a traffic signal, green means “go;” a green plumber means a plumber who is new at the job; a

green triangle with a stem might mean Christmas to some; and the green rectangle on my

computer means “start”.

Signs, the sign systems to which they belong, their function, structures, and properties,

and the way in which signs are used to mediate and convey meaning, are elements common to all

typologies. The study of semiotics goes beyond discussions of signs or symbols as substituting

for something else to include the notion that signs and symbols communicate and construct

meaning. Hervey believes that regardless of the different semiotic contexts and viewpoints, the

correlation between signal and message from any perspective, creates an act of communication:

“The conveying of messages by signals [signs] constitutes the prototype of the phenomenon of

communication” (Hervey, 1982, p. 2).

Page 7: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

7

Traditional approaches to teaching and learning communicate and convey messages

through the signs and sign systems of print literacy and language. Reading and language are

parts of just one sign system among very many in this rapidly changing world. Children in the

twenty-first century enter school with a working knowledge of a rich world of sign systems used

by the latest digital technologies—computers, game boys, play stations, the newest cell phone

devices, X-box and more, that were only starting to come into everyday use a decade ago. The

world is changing and “the ways in which we make meaning are changing. This means that

literacy pedagogy has to change and with it our notion of what can be defined as literate”

(Kalantzis & Cope, 2001, p. 9)

Bridges from the “Old” literacies to the “New” literacies of the twenty-first century were

in construction beginning around the mid 1990’s and those bodies of thought were connected to a

long bridge from the past begun by Dewey in 1934 in “Art and Experience.” Eisner (1994; 2002)

continues Dewey’s argument for moving beyond print-based media and Gardner (1983; 1993)

has long been a powerful voice in advocating multiple pathways to learning. John-Steiner (1997)

explores the world of “cognitive pluralism” and voices such as Leland and Harste (1994), Short

and Harste (1996), and Wright (2003) advocate for the arts as multiple ways of knowing through

a variety of available symbol systems.

The complex, communicative potential of semiotics is important for current approaches to

teaching and learning that include multiple signs and symbol systems. “Semiotics, therefore, is at

once recent—if considered from the viewpoint of the determination of its status and awareness of

its wide-ranging possible applications, and it is also ancient—if its roots are traced back,

following Sebeok, to the theory and practice of ancient medicine” (Petrilli, & Ponzio, 2001, p. 6).

Veivo (2007) states: “If we concede that all thought, values, emotions have to pass through

Page 8: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

8

signs, texts and sign systems in order to be communicated (shared, reflected), then semiosis is

crucial for the…culture and the society” (2007, slide 7).

In 1996, a group of literacy and new media scholars and educators met to discuss the

potential for all signs and sign systems to communicate meaning in ways that would respond to

cultural and societal demands of the twentieth and twenty-first centuries and the growing

significance of rapid changes in globalization, technology, and social diversity and their

accompanying diverse texts and symbol systems (Cope & Kalantzis, 2000). Because this group

of international scholars and educators first met in 1994 in New London, New Hampshire, they

came to be known as the New London Group. As a result of their meeting and discussions in

1996, they published a seminal paper called “A Pedagogy of Multiliteracies: Designing Social

Futures” (New London Group, 1996) in which they conceptualized a pedagogy of

“multiliteracies.” “This landmark article served as a catalyst for global change in literacy

research, policy, curriculum and pedagogy” (Mills, 2006, p. 62) and today still serves as a

catalyst for change in teaching and learning.

The New London Group (1996, 2000) bases their conceptualization of new multiliteracy

pedagogy on two main principals. The term multiliteracies is used to refer to the construction,

understanding, and sharing of meaning not only through language, but through any semiotic

activity or mode of meaning-making, including visual, audio, spatial, and gestural, as well as

combinations of these multiple modes. Secondly, the New London Group states it is their intent

to use the term multiliteracies “to focus on the realities of increasing local diversity and global

connectedness” (1996, p. 4). From this perspective, texts can be created and mediated in various

forms of the English language as represented and used in global cultures, and no form of English

Page 9: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

9

is more important than any other, and no sign system, including language, is more important than

any other meaning-making semiotic system.

I offer an example to support the premise put forward by the New London Group that

traditional notions of being able to read and write printed text are not sufficient for today’s world

and its children. The semiotic linguistic systems must be extended to include meaning-making

and sharing in a wide range of semiotic systems, as my 11 year old son demonstrated to me

recently. He was entered in a tennis tournament and I needed to know the times of his matches in

order to sort out my family’s complicated schedule for that week. I found my son at the family

computer going through his i-tunes music library and transferring his tunes to his i-pod while he

was listening to selections from his music library of over 1000 selections.

I asked him to stop for a minute and look up his match times on the Tennis Manitoba site.

He did the usual clicks to get onto the site, showed me his match times, and started checking the

tennis stats while I wrote down the times of his matches. Every now and then, I would hear a

little “bring, bring” from the computer and Graeme (pseudonym) would perform a few more

clicks, type something, and then return to the tennis stats. After one particular “bring, bring” he

typed the letters B-O-R-E-D and then went back to the tennis site and copied and pasted a picture

of the tennis court he would be playing on that week to send to a friend, while I pondered the

meaning of the letters B-O-R-E-D. Curious, I asked him why he had typed in the word “bored”

moments ago. Graeme responded, “Because Emily was on MSN and she “nudged” me, (hence

the “bring, bring”) and wanted to know what’s up, so I said bored.”

“Graeme,” I responded in turn. “How on earth can you be bored? You’re talking to me

and about five other friends at the same time, you’re listening to music, transferring tunes to your

i-pod, you’re going through tennis stats, and you’re cutting and sending pictures to people—how

Page 10: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

10

can you be bored?” Graeme said, “Well actually Mom, it’s 11—I’m talking to 11 different

friends—I always have 14 people online, but only 11 of them ever say anything.” So I repeated,

“Then how can you possibly be bored?” At that point Graeme became frustrated with my

repeated questioning, threw his hands up in the air, and said, “Because Mom, nothing’s

happening—I’m just sitting here.”

As the New London Group suggested, traditional notions of print text and language were

not sufficient for Graeme to make and communicate sense of his world. Graeme was using

multiple signs and symbol systems, multiple modes of meaning and multiple literacies to

communicate, make meaning, and share meaning. He used linguistic literacies to read the words

on the computer screen and to type messages. He used auditory sign systems to listen to his i-

tunes, to interpret the sound effect of the “bring, bring” and to listen to me. He used visual and

spatial literacies to view images, internet page layouts and screen formats, and to communicate

by copying and sending a picture of the tennis courts he would be playing on to make what he

would be doing later that week meaningful for a friend.

He used numerical sign systems to examine and make meaning from the online tennis

statistics, and he used digital and technological literacies operating his computer and i-pod (and

his game-boy was on a chair beside him). Finally, he used gestural signs towards the end of our

conversation to convey the meaning of his frustration and annoyance to me. He illustrated that

“being literate in today’s society means more than just being able to read and write the written

word. Literacy and learning in new educational environments requires students to be

multiliterate” (Morin, 2006, p. 11). Graeme used a combination of modes identified by the New

London Group in a process “in which the signified (what is meant) is realized through the most

Page 11: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

11

apt signifier (that which is available to give realization to that which is to be meant) in a specific

social context” (Jewitt & Kress, 2003, p. 10).

Jewitt and Kress (2003) note that semiotic theory is the foundation for theories of

multiliteracies or multimodal literacy. However, in the dominant semiotic typologies, systems

for meaning-making and sharing are ready-made and their resources are used in conventional

ways and ways deemed socially appropriate. As Jewitt and Kress observe, “People are not

regarded as having a role in the making or in the reshaping of these resources….People use the

resources, but they do not change them” (2003, p. 10). However, in the context that my son was

using semiotic resources, he was using the most apt semiotic form to give realization to the

meaning he was constructing, communicating, and sharing. He was exemplifying the principal of

“aptness” that underlies the social semiotic multimodal perspective (Jewitt & Kress, 2003).

Jewitt and Kress state that “from a social semiotic perspective, people use the resources

that are available to them in the specific socio-cultural environments in which they act to create

signs, and in using them, they change these resources” (2003, p. 10) so that signs are always

newly made based on the motivation or interest of the maker or user of the sign. Jewitt and Kress

believe that interest of the sign-maker establishes the relationship between the signifier and the

signified. They believe this relationship is realized in three ways: the sign-maker’s interest

determines what is to be signified; the sign-maker decides on the most apt signifier; and the sign-

maker determines how the sign is made most suitable in the circumstances in which it is

communicated.

In this way, any sign system can be used as a signifier if it is judged to be apt for the

circumstance and needs of the sign-maker. Jewitt and Kress observe that there is never an exact

fit between signifier and signified. “In the gap between what they meant to mean and what they

Page 12: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

12

have to use to mean it exists the possibility of the new” (Jewitt & Kress, 2003, p. 13). The role

and potential of multiliteracy approaches lies in that gap where the possibility of the new signs

and sign systems exist for meaning-making and sharing.

New literacy proponents ascribe to a Vygotskian (1978) social nature of literacies in

which meaning-making and sharing is mediated through social context (Hodge & Kress, 1988;

Jewitt & Kress, 2003). An implication of this perspective of social semiotics for teaching and

learning is that one sign system can never fully represent all aspects of what is it teachers or

students wish to represent, as what is chosen to be represented depends upon interest at the time,

the relationship to the audience to whom meaning is being communicated, and upon the social

context of that meaning-making and sharing.

Therefore, early years readers may need more than one symbol system to make sense of

and to communicate and share print literacy meaning. These readers need to explore and

determine the most apt signifier for the circumstance, audience, and social context at any specific

time choosing from a range of available symbol systems. As Wright observes, “Multiliteracy

encompasses a range of expressions and a variety of symbols and symbol systems, not just

words” (2003, p. 148). Multiple sign systems are needed “to encode a variety of meanings” (p.

148).

Wright urges exploration of the rich potential of the sign systems of the arts—music,

drama, visual arts, and dance, to help children make, interpret, and share meaning through

multimodal overlaps and connections across curricula. Print literacy is only one text to be read;

the artistic mediums offer other texts and symbol systems that can be read for meaning-making

and sharing. Early years readers may engage in print literacies through interest and use of

nonprint literacies such as music. A wide body of literature suggests that elements of music,

Page 13: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

13

particularly rhythmic elements, may be a sign system or text that provides opportunities for print

meaning-making and sharing for early years children (Goswami et al., 2002; Lamb & Gregory,

1993; Overy 2003a; 2003b).

Early years children enter school with symbol systems and multiple literacies already in

place through which they can make and share meaning. Millen (1993) observes, “I have found

that traditional reading skills can be improved upon if teachers gain a deeper understanding of

the literacy that many students already have when they come to us” (p. 190). Many early years

students come to school with experiences in musical literacies, through listening to music,

nursery rhymes and songs, and having had nursery rhymes, finger plays, and children’s poetry

read to them. The rhythmic sounds of language and music may be part of their experience bank,

and may be systems that engage and create interest for early years children.

A multiliteracy approach to teaching and learning does not eliminate the need for print

text and print literacies, but it does do away with the “fervently held beliefs about the hierarchies

that exist…among print literacy and other kinds of literacies” (Straw, 1993, p. 9). Because print

literacy may not serve as the starting point for learning, and because all modes are potentially

equally significant for meaning and communication (Jewitt & Kress, 2003), a multiliteracy

approach to teaching and learning does not value or privilege one group of learners over another

but rather embraces differences. If all literacies and semiotic systems are valued for representing,

making, and sharing meaning, then children who might not have success with one semiotic

system, for example print literacy, would have the opportunity to be successful in another, thus

valuing and providing opportunities for success for all early years learners and promoting truly

inclusive education.

Page 14: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

14

Cope and Kalantzis (1997) add strength to this argument by suggesting that rather than

force individuals to share similar values, it is the members’ differences that are actually the

strengths of curriculum design. Members of a community work best when differing interests,

knowledge, experience, culture and language “are respected and used as a source of creativity, or

as a link into the myriad of niches in the world in which the organization has to operate”

(Kalantzis, Cope, & Harvey, 2003, p. 20). Selfe and Hawisher (2004) continue to build the

argument for the importance of multiliteracy or multimodal literacies for all children because

“students from different cultures, races, and backgrounds bring different literacies and different

experiences with literacy to the classroom; focusing so single-mindedly on only one privileged

form of literacy encourages a continuation of the literate/illiterate divide” (p. 232).

Although the theories of the New London Group represent perhaps the most well-known

new literacy pedagogy, several models of new literacies pedagogies have been proposed in recent

years (Kist, 2005). Kist notes the resulting confusion in terminology and the plethora of terms to

describe the new literacies. Kist (2004; 2005) describes approaches to teaching and learning in

multiple media as “new literacies” as does Lankshear and Knobel (2003). The typology of

multiple literacies or multiliteracies is popularized by key scholars and theorists such as Piazza

(1999), Cope and Kalantzis (2000), Gee (1992; 1999; 2001), Kalantzis and Cope (2003),

Unsworth (2001), and Richards and McKenna (2003). Kress (2000a; 2000b) and Jewitt and

Kress (2003) use the term multimodal literacy, and others as noted by Kist (2005) prefer media

literacy, crucial multimedia literacy, or aesthetic literacy. Despite the varying perspectives and

ways in which to describe the new literacies, Mills (2006) notes that a common thread to all these

new literacy pedagogies is the location of this research in the socio-cultural beliefs of literacy

Page 15: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

15

research in the tradition of Vygotsky (1978), Heath (1983; 1986; 1999), and Street (1984; 1995;

1999).

The use of the terms multimodal and multiliteracy have not been without controversy. In

particular the use of the word literacy attached to other words to form terminology such as

computer literacy and emotional literacy has resulted in confusion (Kist, 2005). The new

pedagogies have also been criticized for “watering down” the old basics of reading, writing, and

arithmetic. It is important to note that the ‘old basics’ are not eliminated from this new

curriculum design, but are supplemented by approaches in which a variety of literacies

communicate meaning in familiar and unfamiliar contexts (Kalantzis, Cope, & Harvey, 2003).

In a review of the work and writing of Sebeok’s semiotic perspective as informed by

Pierce, Petrilli and Ponzio (1998) state that in Sebeok’s view the entire universe is made up of

interconnected and interdependent signs within a huge semiotic web. As Pierce states, “the entire

universe is perfused with signs” (Noth, 1990, p. 4). This belief in the interconnected,

interdependent and relational nature of signs is crucial to the semiotic potential of multimodal

experiences for early years readers. Sebeok advocates the notion that signs in our world cannot

be considered separately and independently from other signs. The understanding of any one sign,

and its use for meaning-making and meaning-sharing, is only possible in relation to other signs

used for meaning-making and sharing that act as interpretants of each other and together bring

meaning to the world (Petrilli & Ponzio, 1998). This is the goal of my dissertation research: to

consider the potential of new sign systems through the rhythmic elements of music as an

interpretant of other traditional and nontraditional semiotic systems to bring meaning to the world

of print literacy for all early years children.

Page 16: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

16

Locke’s “Essay Concerning Human Understanding” was one of the starting points to this

discussion. In 1690, he articulated the elements of sign systems today used in multiliteracy

teaching and learning that are so important to all early years learners. Locke states that signs are

used by the mind “for the understanding of things” (meaning-making), or conveying knowledge

to others, “to communicate our thoughts to one another, as well as record them for our own use”

(Locke, 1995, 607-08) (sharing meaning in social contexts).

A declaration by Osolsobe is a favorite statement I discovered in my readings along this

semiotic journey: “One of the most exciting experiences with semiotics is that it brings things

together in a totally unexpected and highly surprising manner” (Borbé, 1979b, p. 1739). This is

the great potential and joy of multimodal, multiliteracy teaching and learning; meaning-making

and sharing occurs in unexpected and surprising ways. As Borbé stated in the opening to the first

volume of his three volume work on semiotics, “There’s no doubt, semiotics is unfolding”

(Borbé, 1979a, p. v). This statement was true centuries ago, nearly three decades ago, and is still

true today. For me and for my research, the semiotic potential of multimodal experiences for

early years readers is unfolding, and in very many unexpected and surprising ways.

Page 17: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

17

References

Agawu, V. K. (1991). Playing with signs. Princeton, NJ: Princeton University Press.

Bearne, E. (2003). Rethinking literacy: Communication, representation and text. Reading

Literacy and Language, 37(3), 98-103.

Borbé, T. (1979a). Semiotics unfolding, vol. 1. Berlin: Mouton Publishers.

Borbé, T. (1979b). Semiotics unfolding, vol. 1. Berlin: Mouton Publishers.

Cope, B., & Kalantzis, M. (1997). Productive diversity: A new Australian approach to work and

management. Sydney, Australia: Pluto.

Cope, B., & Kalantzis, M. (Eds.). (2000). Multiliteracies: Literacy learning and the design of

social futures. London: Routledge.

Danesi, M. (2004). Messages, signs, and meanings: A basic textbook in semiotics and

communication theory. Toronto: Canadian Scholars’ Press Inc.

Dewey, J. (1934). Art as experience. New York: Minton & Balch.

Eco, U. (1976). Theory of semiotics. Bloomington, IL: Indiana University Press.

Eisner, E. (1994). Cognition and curriculum reconsidered (2nd ed.). New York: Teachers College

Press.

Eisner, E. (2002). The arts and the creation of mind. New Haven, CT: Yale University Press.

Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic

Books.

Gardner, H. (1993). Multiple intelligences: The theory into practice. New York: Basic Books.

Gee, J. (1992). The social mind: Language, ideology, and social practice. New York: Bergin &

Garvey.

Page 18: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

18

Gee, J. (1999). Social linguistics and literacies: Idology in discourses (2nd Ed.). Philadelphia:

Falmer Pres, Taylor & Francis.

Gee, J. (2001). Literacies, schools, and kinds of people: Educating people in the new capitalism.

In M. Kalantzis & B. Cope (Eds.), Transformations in language and learning:

Perspectives on multiliteracies. Australia: Common Ground Publishing.

Goswami, U., Thomson, J., Richardson, U., Stainthorp, R., Hughes, D., Rosen, S., & Scott, S. K.

(2002). Amplitude envelope onsets and developmental dyslexia: A new hypothesis.

Proceedings of the National Academy of Sciences, USA, 99, 10911-10916.

Hardwick, C. (Ed.). (1977). Metaphors and multiples: Representation, the arts, and history. Phi

Delta Kapan, 78, 387-394.

Harste, J. C., Woodward, V., & Burke, C. Language stories and literacy lessons. Portsmouth,

NH: Heinemann, 1984.

Heath, S. (1983). Ways with words: Language, life and work in communities and classrooms.

Canbridge: Cambridge University Press.

Heath, S. (1986). Critical factors in literacy development. In S. DeCastell, A. Luke, & K. Egan

(Eds.), Literacy, society and schooling. Cambridge: Cambridge University Press.

Heath, S. (1999). Literacy and social practice. In D. Wagner, R. Venezky, & B. Street (Eds.),

Literacy: An international handbook. Boulder, CA: Westview Press.

Hervey, S. (1982). Semiotic perspectives. London, England: George Allen & Unwin.

Hodge, R., & Kress, G. R. (1988). Social semiotics. Cambridge, UK: Polity Press.

Innis, R. E. (1985). Semiotics: An introductory anthology. Bloomington, IN: Indiana University

Press.

Page 19: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

19

Jewitt, C. (2003). Re-thinking assessment: Multimodality, literacy and computer-mediated

learning. Assessment in Education, (10)1, 83-102.

Jewitt, C., & Kress, G. (2003). Multimodal literacy. New York: Peter Lang.

John-Steiner, V. (1997). Notebooks of the mind: Explorations of thinking (Rev. ed.). New York:

Oxford University Press.

Kalantzis, M. & Cope, B. (Eds.). (2001). Transformations in language and learning:

Perspectives on multiliteracies. Australia: Common Ground Publishing.

Kalantzis, M., Cope, B., & Harvey, A. (2003). Assessing multiliteracies and the new basics.

Assessment in Education, (10)1, 15-26.

Kist, W. (2004). The new literacies movement. Independent School, (63)4, 1-5.

Kist, W. (2005). New literacies in action: Teaching and learning in multiple media. New York:

Teachers College Press.

Kress, G. (1997). Literacy in the new media age. London: Routledge.

Kress, G. (2000a). Design and transformation: New theories of meaning. In B. Cope & M.

Kalantzis (Eds.), Multiliteracies: Literacy learning and the design of social futures (pp.

153-161). Melbourne: Macmillan.

Kress, G. (2000b). Multimodality. In B.Cope & M. Kalantzis (Eds.), Multiliteracies: Literacy

learning and the design of social futures (pp. 182-202). Melbourne: Macmillan.

Kress, G., Jewitt, C., Ogborn, J., & Tsatsarelis, C. (2001). Multimodal teaching and learning:

The rhetorics of the science classroom. New York: Continuum Press.

Lamb, S. J., & Gregory, A. H. (1993). The relationship between music and reading in beginning

readers. Educational Psychology, 13, 13-27.

Page 20: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

20

Lankshear, C., & Knobel, M. (2003). New literacies: Changing knowledge and classroom

learning. Philadelphia, PA: Open University Press.

Leland, C. H., & Harste, J. C. (1994). Multiple ways of knowing: Curriculum in a new key.

Language Arts, 71, 337-345.

Locke, J. (1995). An essay concerning human understanding. Amherst, NY: Prometheus Books.

Maeder, C., Fisher, O., & Herlofsky, W. (2003). Outside-in—Inside-out: Iconicity in language

and literature 4. Amsterdam, The Netherlands: John Benjamin Publishing Company.

Millen, J. (1993). Print and televisual literacies: Uneasy companions. In S. B. Straw & D. Bogdan

(Eds.), Constructive readings: Teaching beyond communication (p. 190-204).

Portsmouth, NH: Boynton/Cook Publishers, Inc.

Mills, K. (2006). Discovering design possibilities through a pedagogy of multiliteracies. Journal

of Learning Design, (1)3, 61-72.

Monelle, R. (1992). Linguistics and semiotics in music. Chur, Switzerland: Harwood Academic

Publishers.

Morin, F. (2006). Redefining literacy. Manitoba Association of School Superintendents Journal,

Spring 2006, 11-14.

New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard

Educational Review, (66)1, 60-92.

New London Group. (2000). A pedagogy of multiliteracies: Designing social futures. In B. Cope

& M. Kalantzis (Eds.), Multiliteracies: Literacy learning and the design of social futures

(pp. 9-37). Melbourne, Macmillan.

Noth, W. (1990). Handbook of semiotics. Bloomington, IN: Indiana University Press.

Page 21: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

21

Overy, K. (2003a). Dyslexia and music: From timing deficits to music intervention. Annals of the

New York Academy of Sciences, 999, 497-505.

Overy, K. (2003b). Dyslexia and music: measuring musical timing skills. Dyslexia, 9, 18-36.

Packwood, N. (1996). Signs: An introduction to semiotics: Review by A. Nicholas Packwood

[Electronic version]. Journal of World Anthropology, (1)3. Retrieved February 9, 2007,

from http://www.wings.buffalo.edu/research/anthrogis/JWA/V1N3/

Petrilli, S., & Ponzio, A. (1998). Signs of research on signs. Special Issue of Semiotische

Berichte, (22)3, 4. Retrieved August 16, 2006, from

http://www.augustoponzio.com/preface.htm

Petrilli, S., & Ponzio, A. (2001). Thomas Sebeok and the signs of life. Cambridge, UK: Icon

Books Ltd.

Piazza, C. (1999). Multiple forms of literacy: Teaching literacy and the arts. Upper Saddle River,

NJ: Merrill/Prentice Hall.

Richards, J. C., & McKenna, M. C. (2003). Integrating multiple literacies in K-8 classrooms:

Cases, commentaries, and practical applications. Mahwah, NJ: Lawrence Erlbaum

Associates, Publishers.

Saussure, F de. (1992). The linguistic sign. In M. Danesi, & D. Santeramo (Eds.), Introducing

semiotics: An anthology of readings (pp. 29-46). Toronto: Canadian Scholars’ Press.

Sebeok, T. A. (1994). Signs: An introduction to semiotics. Toronto: University of Toronto Press.

Selfe, C., & Hawisher, G. (2004). Literate lives in the information age. Mahwah, NJ: Lawrence

Erlbaum Associates.

Short, K., & Harste, J. (with Burke, C.). (1996). Creating classrooms for authors and inquirers

(2nd ed.). Portsmouth, NH: Heinemann.

Page 22: Semiotic Potential of Multimodal Experiences for Early ... · and profuse semiotic typologies including communication semiotics, ... green triangle with a stem might mean Christmas

22

Solomon, J. (1988). The signs of our times. New York: Harper & Row, Publishers.

Straw, S. B., & Bogdan, D. (1993). Constructive reading: Teaching beyond communication.

Portsmouth, NH: Boynton/Cook Publishers, Inc.

Street, B. (1994). Literacy in theory and practice. Cambridge: Cambridge University Press.

Street, B. (1995). Social literacies: Critical approaches to literacy in development, ethnography

and education. London: Longman.

Street, B. (1999). The meaning of literacy. In D. Wagner, R. Venezky, & B. Street (Eds.),

Literacy: An international handbook. Boulder, CO: Westview Press.

Unsworth, L. (2001). Teaching multiliteracies across the curriculum: Changing contexts of text

and image in classroom practice. Buckingham, UK: Open University Press.

Veivo, H. (2007). Semiotics of literature. Retrieved January 23, 2007, from University of

Helsinki, Semiotics of Literature Web site:

http://www.helsinki.fi/taitu/semiotiikka/kevat2007/Lecture_1.ppt

Vygotsky, L. (1978). Mind in society: The development of higher psychological processes.

London: Harvard University Press.

Wright, S. (2003). The arts, young children, and learning. Boston, MA: Pearson Education, Inc.