www.arts-journal.com

The InternationalJOURNALoftheARTS

IN SOCIETY

Volume 4, Number 4

Towards a Model of Intersemiotic Translation

Daniella Aguiar and Joao Queiroz

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY http://www.arts-journal.com First published in 2009 in Champaign, Illinois, USA by Common Ground Publishing LLC www.CommonGroundPublishing.com. © 2009 (individual papers), the author(s) © 2009 (selection and editorial matter) Common Ground Authors are responsible for the accuracy of citations, quotations, diagrams, tables and maps. All rights reserved. Apart from fair use for the purposes of study, research, criticism or review as permitted under the Copyright Act (Australia), no part of this work may be reproduced without written permission from the publisher. For permissions and other inquiries, please contact <cg-support@commongroundpublishing.com>. ISSN: 1833-1866 Publisher Site: http://www.Arts-Journal.com THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY is peer-reviewed, supported by rigorous processes of criterion-referenced article ranking and qualitative commentary, ensuring that only intellectual work of the greatest substance and highest significance is published. Typeset in Common Ground Markup Language using CGCreator multichannel typesetting system http://www.commongroundpublishing.com/software/

Towards a Model of Intersemiotic TranslationDaniella Aguiar, State University of Rio de Janeiro (UERJ), Rio deJaneiro, BrazilJoao Queiroz, Federal University of Juiz de Fora (UFJF), Minas Gerais,Brazil

Abstract: The phenomenon of intersemiotic translation (IT) represents a special creative domain oflanguage procedures and practices. Hypothetically it involves a radical change of habits of interpret-ation and new forms of sign manipulation. Despite its theoretical relevance, and in spite of the frequencein which it is practiced, the phenomenon remains virtually unexplored in terms of conceptual modeling,especially from a semiotic perspective. Indeed, the phenomenon of IT is difficult to characterize andcompare with analogous phenomena (interlingual translation). As it involves systems of rather distinctnature, its analysis creates additional difficulties in any theoretical approach compromised with thelogic of semiotic processes. This work proposes an approach based on Charles S. Peirce’s model ofsign process, to provide a preliminary conceptual framework to the phenomena, emphasizing hierarch-ical properties and aspects. One of the consequences of our approach is the importance ascribed tothe materiality and dynamic involved in IT, prioritizing the semiotic properties of hierarchical relationsbetween the source and the target signs.

Keywords: Intersemiotic Translation, Translation, Semiosis, C.S. Peirce

INTERSEMIOTIC TRANSLATION (IT) was defined by Roman Jakobson (2000[1959]: 114) as ‘transmutation of signs’ – ‘an interpretation of verbal signs by meansof signs of non verbal sign systems’. Despite its theoretical relevance, and in spite ofthe frequence in which it is practiced, the phenomenon remains virtually unexplored in

terms of conceptual modeling, especially from a semiotic perspective.For Gorlée (2007: 347), creative sign transmutation involves the ‘reconstruction’ of an

artwork into a distinct semiotic system, creating a sophisticated collection of interconnectedsigns (see also Plaza, 1987). The modalities of translation proposed by Jakobson, accordingto Gorlée (1994: 147-168, 1997: 240-244, 2005: 34-35), are related to the notion of translationin an ‘extra-linguistic horizon’. This leads us to a general acceptance of translations of textsof all kinds, taking away from the term its exclusive allusion to linguistic material (see Petrilli,2003).

The processes are observed in several systems including: literature and cinema (HermanMelville > John Huston; William Shakespeare > Orson Welles; Vladimir Nabokov > StanleyKubrick), literature and comics (Herman Melville > Bill Sienkiewicz), poetry and dance(Theóphile Gautier > Michel Fokine; Stéphane Mallarmé > Nijinsky). Many other examplescould be mentioned in several sign systems as theater, sculpture, music, painting, video, andso on.

The main methodological difficulty is related to the comparison between radically differentsemiotic systems. As we know, a translation is not committed only to ‘semantics’ (see

The International Journal of the Arts in SocietyVolume 4, Number 4, 2009, http://www.arts-journal.com, ISSN 1833-1866© Common Ground, Daniella Aguiar, Joao Queiroz, All Rights Reserved, Permissions:cg-support@commongroundpublishing.com

Campos, 1972), to which ‘meaningful dimensions’ of semiotic processes are usually attributed.It seems theoretically natural to describe an interlingual translation by establishing directcorrelations between comparable semiotic layers of organization – morphological-morpho-logical, phonetic-phonetic, rhythmic-rhythmic (see Jakobson & Pomorska, 1985). However,an IT does not exhibit the same principle of corresponding layers (see Plaza, 1987).

Here we propose an approach based on Charles S. Peirce’s model of sign process, toprovide a preliminary conceptual framework to the phenomena emphasizing hierarchicalproperties and aspects.

Intersemiotic Translation ModelOur approach is based on two basic premisses: (i) translation is fundamentally a semioticoperation process (semiosis) (Hodgson, 2007; Petrilli, 2003; Stecconi, 1999; Plaza, 1987);(ii) semiosis is a multi-layered process (Queiroz & El-Hani, 2006, 2004).

Intersemiotic Translation as Semiosis or the ‘Action of Sign’Peirce defined semiosis as an irreducible triadic relation between a Sign, its Object and itsInterpretant — we will hereafter refer to this sign triad as S, O, and I (CP 2.171, CP 2.274).1

According to Peirce, any description of semiosis involves a relation constituted by three ir-reducibly connected terms, which are its minimal constitutive elements (MS 318:81; CP2.242). In his words:

‘A Sign may be defined as a Medium for the communication of a Form. [...] As a me-dium, the Sign is essentially in a triadic relation, to its Object which determines it, andto its Interpretant which it determines. [...] That which is communicated from the Objectthrough the Sign to the Interpretant is a Form; that is to say, it is nothing like an existent,but is a power, is the fact that something would happen under certain conditions’ (MS793:1-3; EP2, p. 544, n. 22).

According to De Tienne (2003), Peirce is emphatic when he observes that form is nothinglike a thing. It is something that is incorporated into the Object (EP 2.544, n. 22) as a habit,an ‘action rule’ (CP 5.397, CP 2.643), a ‘disposition’ (CP 5.495, CP 2.170), or a ‘real poten-tial’ (EP 2.388).

There are some important consequences from Peirce’s theory of sign process. This modelof semiotic operation describes a phenomenon as essentially triadic, dynamic, interpreter-dependent, and materially extended (embodied) (see Queiroz & Merrell, 2009, 2006). Em-phatically, a semiotic process, consequently an IT, is NOT a bi-lateral relation.

Intersemiotic Translation as a Multi-layered ProcessAnother presupposition is related to what we call ‘layer of organization or description level’:if semiotic processes are multi-layered (multi-level systems), as we have argued in other papers

1 We shall follow the practice of citing from the Collected Papers of Charles Sanders Peirce (Peirce, 1931-35, 1958)by volume number and paragraph number, preceded by ‘CP’; the Essential Peirce by volume number and pagenumber, preceded by ‘EP’. References to the micro¬film edition of Peirce's papers (Harvard University) will beindicated by ‘MS’, followed by the manuscript number.

204

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

(see Queiroz & El-Hani, 2006, 2004), a translation is a relation between multi-layered pro-cesses, and an IT can be described as a multi-hierarchical process of relation between semi-independent layers of organization. The layers are coordinated in terms of mutual constraints.In this sense, although we can describe the ‘scenic dance space’, for instance, without refer-ence to ‘movement dynamic morphology’, in fact they are mutually constraining each otherin a dance choreography.

IT operates on different layers, selecting relevant aspects from the source and recreatingthem into the target (see Eco, 2007: 59 and Campos, 1972 about poetic translation). In thisway, certain layers have their relevant properties selected and translated into new materialsand processes. For example, from literature to dance, linguistic layers (rhythmic, prosodic,syntactic, or psychological ambience) are translated into dynamic of movement, organizationof space, light design, costumes, scenography, etc. Notably, a ‘mapping of correlations’cannot be easily established between layers of different nature (different semiotic systems).If a translation from a literary work into a dance choreography results in very different ma-terials and structures, how is it possible to compare ‘semiotics source and target’? In anycase, possibilities of conceptual mapping (probably non-univocal) between different systemsand levels should be provided.

Triadic Translation ModelAn important consequence related to our premises, as was specified above, indicates that atranslation is a triadic (S-O-I) relation, not a dyadic-bilateral one.

There are two competing model possibilities:

• The sign is the semiotic source (translated work). The object of the translated sign is theobject of the semiotic-source and the interpretant (produced effect) is the semiotic target(translator sign). (Figure 1).

Figure 1: Triadic Relation in which the Sign is the Translated Work, the Object of the Signis the Object of the Work, and the Interpretant is the Translator Sign

205

DANIELLA AGUIAR, JOAO QUEIROZ

• The sign is the semiotic-target (translator sign). The object of the sign is the semioticsource (translated work) and the interpretant is the effect produced on the interpreter(interpretant). (Figure 2).

Figure 2: Triadic Relation in which the Sign is the Target, the Object of the Sign is theTranslated Work, and the Interpretant is the Interpreter

In an effort to a better explanation of model possibilities, we will exemplify them with theSpider-Man’s comic-film translation. According to the first possibility, we could replace thesign-object-interpretant triad with the comic book - comic book object - film relation. In thiscase, the sign is the Spider-Man comic book; the object is the Spider-Man comic book objectthat, in a simplified explanation, should be the overcoming of a “spider-man hybrid hero”;and the interpretant, the effect, is a Spider-Man film. According to the second model, thesign-object-interpretant triad could be replaced with the film - comic book - effect on theaudience. Hence, the sign is a Spider-Man film; the object is the Spider-Man comic book,and the interpretant is the effect of the film on the audience.

According to the process described above, the ‘form’ communicated from the object tothe effect (interpretant) and produced by means of the sign is different in each version. Howcan these differences be helpful? In further works, we should speculate about how those al-ternatives provide insights about the phenomenon examined.

At this point, we insert the hierarchical schema to the triadic process considering the firstalternative model. According to this version, a (semiotic) relation of translation betweenmulti-structured processes is established in terms of irreductible triadic relation. In an exampleof translation from literature to dance, the interpretant (costume, rhythm, movement) is de-termined by the object (object of semiotic target), through the sign (history, pragmatic,syntax) (Figure 3):

206

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

Figure 3: Translation Model from Literature to Dance that Includes the Notion of ‘Layer ofOrganization or Description Level’

The second version provides us another perspective of the phenomenon, with focus on thereader. Including the hierarchical relations schema to this version, we will have the interpretant(the effect on the interpreter) determined by the object (history, pragmatic, syntax), throughthe sign (costume, rhythm, movement) (Figure 4).

207

DANIELLA AGUIAR, JOAO QUEIROZ

Figure 4: Translation Model (II) from Literature to Dance that Includes the Notion of ‘Layerof Organization or Description Level’

ConclusionIT represents a domain of new language processes because it tends to produce differenthabits of sign manipulation and interpretation. Beyond that, it involves a pragmatic view ofthe processes resulting from direct comparison of very different semiotic systems. Neverthe-less, there are small amounts of theoretical works systematically produced about the phe-nomenon.

Indeed, the phenomenon of IT is difficult to characterize and compare with analogousphenomena (e.g. interlingual translation). As it involves systems of rather distinct nature,its analysis creates additional difficulties in any theoretical approach compromised with thelogic of semiotic processes.

One of the consequences of our approach is the importance ascribed to the materialityand dynamic involved in IT, prioritizing the semiotic properties of hierarchical relationsbetween the source and the target signs. The partial results exhibited constitute a preliminaryattempt toward modeling IT.

208

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

AcknowledgmentsWe thank Floyd Merrell and Lucia Naser for helpful comments during the preparation ofthis manuscript. J.Q. thanks the State of Minas Gerais Research Foundation (FAPEMIG).

ReferencesCampos, Haroldo. A arte no horizonte do provável. São Paulo: Perspectiva, 1972.De Tienne, Andre. “Learning qua semiosis.” S.E.E.D. 3 (2003): 37-53.Eco, Umberto. Quase a Mesma Coisa . Rio de Janeiro: Record, 2007.Gorlée, Dinda L. Semiotics and the Problem of Translation, With Special Reference to the Semiotics

of Charles S. Peirce. Amsterdam/Atlanta: Rodopi, 1994.———. “Intercode translation: Words and music in opera.” Target 9 (2) (1997): 235-270.———. “Singing on the breath of God.” In Song and Significance: Virtues and Vices of Vocal

Translation , ed. Dinda L. Gorlée. Amsterdam/New York: Rodopi, 2005.———. “Bending back and breaking.” Symploke 15 (1-2). (2007): 341-352.Hodgson, Robert. “Semiotics and Bible translation.” Semiotica 163 (1/4) (2007): 37-53.Jakobson, Roman. “On linguistic aspects of translation.” In The Translation Studies Reader , ed.

Lawrence Venuti. London/New York: Routledge, 2000 (1959).Jakobson, Roman, and Kristina Pomorska. Diálogos . São Paulo: Cultrix, 1985.Peirce, Charles S. The Essential Peirce, Selected Philosophical Writings . (Vol. 1 ed. by N. Houser &

C. Kloesel; Vol 2 ed. by the Peirce Edition Project). Bloomington and Indianapolis: IndianaUniversity Press, EP1 1992, EP2 1998. (Quoted as EP).

Peirce, Charles S. The Collected Papers of Charles Sanders Peirce . Electronic edition reproducingVols. I–VI [C. Hartshorne & P. Weiss (eds.), Cambridge: Harvard University Press,1931–1935]; Vols. VII—VIII [A. W. Burks (ed.), same publisher, 1958]. Charlottesville:Intelex Corporation, 1931–1935. (Quoted as CP).

Peirce, Charles S. Annotated Catalogue the Papers of Charles S. Peirce . (ed.) R.S. Robin. Massachu-setts: The University of Massachusetts Press, 1967. (Quoted as MS).

Petrilli, Susan. “Translation and Semisis.” In Translation Translation , ed. Susan Petrilli. Amster-dam/New York: Rodopi, 2003.

Plaza, Julio. Tradução Intersemiótica. São Paulo: Perspectiva, 1987.Queiroz, João, and Charbel El-Hani. “Towards a multi-level approach to the emergence of semiosis.”

Technical Report DCA-FEEC 04 (07) (2004): 1-21.———. “Semiosis as an emergent process.” Transaction of C.S.Peirce Society 42 (1) (2006): 78-116.Queiroz, João, and Floyd Merrell. “Semiosis and pragmatism: toward a dynamic concept of meaning.”

Sign System Studies 34 (1) (2006): 37-66.Queiroz, João, and Floyd Merrell. “On Peirce´s pragmatic notion of semiosis – a contribution for the

design of meaning machines.” Minds & Machines 19 (2009): 129-143.Stecconi, Ubaldo. “Peirce’s semiotics for translation.” In Fidelity and Translation: Communicating

the Bible in New Media , ed. Paul A. Soukup and Robert Hodgson. New York/Franklin:American Bible Society/Sheed and Ward, 1999.

About the AuthorsDaniella AguiarDaniella Aguiar is a PhD student on Comparative Literature at the Graduate Studies Programon Literature, State University of Rio de Janeiro (UERJ), Brasil. She obtained her bachelordegree in Dance from Anhembi Morumbi University, São Paulo, Brasil, in 2004; master

209

DANIELLA AGUIAR, JOAO QUEIROZ

degree in Dance from Graduate Studies Program in Dance, from Federal University of Bahia(UFBA), Brasil, in 2008. Her main research interests are in Dance Theory, Semiotics, Cog-nitive Science, Cognitive Aesthetics, and Comparative Literature. Her main research topicsare (i) dance techniques, (ii) cognitive artifacts, (iii) intersemiosis and (iv) intersemiotictranslation. She is also involved in creative and educational projects in dance as choreographer,dancer and teacher.

Joao QueirozJoao Queiroz is a professor at the Institute of Arts and Design and at the Graduate Programin Communication, Federal University of Juiz de Fora, Minas Gerais, Brazil. He earned aPh.D. in Communication and Semiotics from the Catholic University of Sao Paulo (PUCSP), and a post-doc fellowship in Intelligent Systems at the School of Electrical and ComputerEngineering (FEEC-DCA), State University of Campinas(UNICAMP), and in Philosophyof Biology at the Institute of Biology, Federal University of Bahia (UFBA). He is the Editor-in Chief of the International Journal of Signs and Semiotic Machines (An Official Publicationof the Information Resource Management Association, New in 2011). His research interestsinclude: intersemiotic translation; emergence and evolution of semiotic complexity; emergenceof symbolic based communication; cognitive technologies; Peirce’s semiotic and pragmatism.

210

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

EDITORS Mary Kalantzis, University of Illinois at Urbana-Champaign, USA. Bill Cope, University of Illinois at Urbana-Champaign, USA. EDITORIAL ADVISORY BOARD Robyn Archer, Performer and Director, Paddington, Australia.

Mark Bauerlein, National Endowment for the Arts, Washington, D.C., USA.

Tressa Berman, BorderZone Arts, Inc., San Francisco, USA; University of Technology, Sydney, Australia; San Francisco Art Institute, San Francisco, USA.

Judy Chicago, Artist and Author, New Mexico, USA.

Nina Czegledy, University of Toronto, Toronto, Canada; Concordia University, Montreal, Canada.

James Early, Smithsonian Institution, Washington, D.C., USA.

Mehdi Faridzadeh, International Society for Iranian Culture (ISIC), New York, USA, Tehran, Iran.

Jennifer Herd, Queensland College of Art, Griffith University, Brisbane, Australia.

Fred Ho, Composer and Writer, New York, USA.

Andrew Jacubowicz, University of Technology, Sydney, Australia.

Gerald McMaster, Curator, Art Gallery of Ontario, Toronto, Canada.

Mario Minichiello, Birmingham Institute of Art and Design, Birmingham, UK.

Fred Myers, New York University, New York, USA.

Darcy Nicholas, Porirua City Council, Porirua, New Zealand.

Daniela Reimann, Institute of Media in Education, University of Education, Freiburg, Germany; University of Art and Industrial Design, Linz, Austria.

Arthur Sabatini, Arizona State University, Phoenix, USA.

Cima Sedigh, Sacred Heart University, Fairfield, USA.

Peter Sellars, World Arts and Culture, University of California, Los Angeles, USA.

Ella Shohat, New York University, New York, USA.

Judy Spokes, Arts Victoria, South Melbourne, Australia.

Tonel (Antonio Eligio), Artist and Art Critic, Havana, Cuba.

Marianne Wagner-Simon, World Art Organization, Berlin, Germany.

Please visit the Journal website at http://www.Arts-Journal.com for further information about the Journal or to subscribe.

THE UNIVERSITY PRESS JOURNALS

Creates a space for dialogue on innovative theories and practices in the arts, and their inter-relationships

with society. ISSN: 1833-1866

http://www.Arts-Journal.com

Explores the past, present and future of books,

publishing, libraries, information, literacy and learning in the information society.

ISSN: 1447-9567 http://www.Book-Journal.com

Examines the meaning and purpose of ‘design’ while also speaking in grounded ways about the task of design and the use of designed artefacts and

processes. ISSN: 1833-1874

http://www.Design-Journal.com

Provides a forum for discussion and builds a body of knowledge on the forms and dynamics of difference

and diversity. ISSN: 1447-9583

http://www.Diversity-Journal.com

Maps and interprets new trends and patterns in

globalisation. ISSN 1835-4432

http://www.GlobalStudiesJournal.com

Discusses the role of the humanities in contemplating the future and the human, in an era otherwise dominated by scientific, technical and economic

rationalisms. ISSN: 1447-9559

http://www.Humanities-Journal.com

Sets out to foster inquiry, invite dialogue and build a body of knowledge on the nature and future of

learning. ISSN: 1447-9540

http://www.Learning-Journal.com

Creates a space for discussion of the nature and future of organisations, in all their forms and

manifestations. ISSN: 1447-9575

http://www.Management-Journal.com

Addresses the key question: How can the institution

of the museum become more inclusive? ISSN 1835-2014

http://www.Museum-Journal.com

Discusses disciplinary and interdisciplinary approaches to knowledge creation within and across the various social sciences and between the social,

natural and applied sciences. ISSN: 1833-1882

http://www.Socialsciences-Journal.com

Draws from the various fields and perspectives through which we can address fundamental

questions of sustainability. ISSN: 1832-2077

http://www.Sustainability-Journal.com

Focuses on a range of critically important themes in the various fields that address the complex and

subtle relationships between technology, knowledge and society.

ISSN: 1832-3669 http://www.Technology-Journal.com

Investigates the affordances for learning in the digital

media, in school and throughout everyday life. ISSN 1835-2030

http://www.ULJournal.com

Explores the meaning and purpose of the academy in times of striking social transformation.

ISSN 1835-2030 http://www.Universities-Journal.com

FOR SUBSCRIPTION INFORMATION, PLEASE CONTACT

subscriptions@commonground.com.au