MUSICAL INSTRUMENTS AS SCORES: A HYBRID APPROACH

Enrique TomasUniversity of Art and Design LinzEnrique.Tomas@ufg.at

ABSTRACT

The development of new approaches to instrumentality dur-ing the decade of 1960 contributed to the dual perceptionof instruments as scores. For many performers, the instru-ment became the score of what they played. This artis-tic hybridization carries substantial questions about the na-ture of our scores and about the relationships among in-struments, performers and musical works. This paper con-textualizes the historical origins of this instrumental devel-opment within Drucker’s theory of performative material-ity. Then we examine the nature and notational scheme ofthis type of scores making use of the concept of inherentscore. Finally, through the analysis of two examples (tan-gible scores and choreographic objects) and the notions ofaffordance and constraint, a compositional framework forshaping the inherent instrument score is presented.

1. INTRODUCTION

1.1 Inherent Scores: Origins of a Form

The idea that a musical instrument can be considered amusical score too, or that the instrument is the score hascome to the academic discussion partly in the field of in-terfaces for musical expression [1], music notation [2] andperformance of electronic music [3]. Certainly, when aperformer approaches a musical instrument a number oflimitations or constraints will be revealed. These charac-teristics of the instrument are often considered a score invirtue of its property of shaping the musical work. Thefollowing section contextualizes historically the origins ofsome musical practices fully supporting this idea.

It is widely considered that the creative interpretation ofmusical instruments as scores has its roots in the Sixties.Composers like Gordon Mumma, David Tudor or DavidBehrman built electronic music instruments that, once con-figured, can afford enough performative potentials to re-veal a musical work. Alvin Lucier [4] describes how withinmany of the works produced by the Sonic Arts Union therewere no scores to follow; the scores were inherent in thecircuitry. In David Behrman’s Runthrough (1968) an un-defined number of performers interact with the instrumentby illuminating parts of a light sensitive audio mixer. Con-ceived as an improvisational piece, Behrman allowed am-

Copyright: c©2016 Enrique Tomas et al. This is an open-access article distributed

under the terms of the Creative Commons Attribution 3.0 Unported License, which

permits unrestricted use, distribution, and reproduction in any medium, provided

the original author and source are credited.

ple time for the possibilities offered by his circuit to un-fold and explore the acoustics of the actual room used. InRunthrough performers are not provided with instructionsabout the type of sound sources to use, their durations, sec-tions of the piece, etc. The general rules for performing aredelegated to player’s musical exploration.

In the same year, David Tudor composed Rainforest (1968)(Figure 1). In this work, a set of sculptural speakers aresuspended in the installation space and act as unique reso-nant loudspeakers with sound emanating directly from thesculptural objects (each having a unique sound source).In Rainforest the compositional idea is that if you sendsound through materials, the resonant nodes of the mate-rials are released. It is a kind of physical filter. Visitorsare encouraged to wander around and physically interactwith the work. Tudor’s notation of the composition is, ina deliberate way, only a circuit diagram (Figure 1). Likein Runthrough, Rainforest can be played without furtherinstructions about durations, sound sources or number ofsculptural speakers.

It is during this historical decade when an intense re-search on alternatives to traditional musical notation wasproduced. A seminal example is Notations (1969), a printedcompendium of musical notation edited by John Cage andAlison Knowles. It is remarkable that among 269 composi-tions, in Notations we only find three musical works mak-ing use of circuit schematics as notation: Gordom Mumma’sMesa (1966), Max Neuhaus’s Max-Feed (1966) and FredicRjewski’s Piece with Projectors and Photocell Mixer (1966).Certainly, if circuits are configured in a specific way fortheir artistic use, the role of the composer, at an equidis-tant point among designers, composers and performers,would start with the configuration of the technical systembehind the actual instrument. Indeed, performing becomesthe creative exploration in freedom of the musical affor-dances, musical reactions or acoustic relations to the phys-ical space performed, without the need of any kind of ded-icated musical composition.

When Lucier exposes that ”the score is inherent in thecircuitry” we are facing the origins of a new composi-tional practice, often known as composing inside electron-ics. And in this sense it constitutes a new way of under-standing instrumentality. Performers would not need anexternal cause, a precondition to play the instrument likein the case of traditional scores. The musical work wouldnot only be only defined by the instrument, but more im-portantly, by the act of playing the instrument. The per-former’s role would be to reveal instances of the musicalwork inherently integrated in the circuitry. This type ofembedded-in-the-instrument scores we will call inherent

Figure 1. David Tudor’s Rainforest (1968).

scores.

Through the concept of inherent scores we can better ana-lyze the complex and mediated relationship between com-posers, performers and their instruments, especially in thecase of electronic musical instruments. It has been oftenstated that electronic music instruments are open-endedsystems [25]. Many times it is difficult to define wherethe electronic instrument ends and the composition starts.Certainly, for defining the instrument it is necessary to im-plement some input to output mapping strategies. But nor-mally these strategies are fully affected by the character-istics of the composition to which the instrument is ded-icated. Remarking this observation, Schnell and Battierintroduced the concept of composed instruments [5]. Thisterm serves to explain how our digital instruments equally”carry the notion of an instrument as that of a score”, inthe sense of determining various aspects of a musical workitself. This is coherent with the fact that during the tech-nical implementation of the instrument, being hardware orsoftware, we often incorporate many ideas of compositioninto the programmed system.

A substantial question to this new musical practice wouldbe if it resulted from a compositional, instrumental or tech-nical development. We can illustrate an answer throughthe analysis of David Tudor’s Bandoneon! (1966). Inthe words of its author, Bandoneon! is a combine of pro-grammed audio circuits, moving loudspeakers, TV images,and lighting, all controlled through the live sound of a ban-doneon played by Tudor. From the program notes of thiswork’s premiere we read that ”Bandoneon! uses no com-positional means, since it composes itself out of its owncomposite instrumental nature” [6]. Kuivila also assertsthat we were in fact facing a new way of understandinginstrumentality. In these self-composed instruments, Tu-dor acts as the interpreter and performer of a compositionthat composes itself out of these constituent parts. Or us-ing Lucier’s arguments, the composition is created fromthe inherent scores that can be found in the structural el-ements of a particular electronic configuration. This con-cept carries an extraordinary rendition: the acceptance thatan electronic instrument is an entity that can display itself

without the need of a composer or a composition. Probablythe most important characteristic of these inherent scores isthat they can reveal or display themselves to their perform-ers only at the exact moment of being performed.

Therefore, we can now assert that the origins of this newmusical practice, under the influence of electronics andthe germinating attitudes of post-modernism 1 , trace theirroots mainly on the appearance of a new approach to in-strumentality. Thus, it does not mainly rely on the mereevolution of an existing compositional practice. The intro-duction of electronic components in composition definitelychanged the understanding of what until that moment wasdefined as ”playing”. Many of this electronic circuitry wasable to synthesize sound or modify sound without the needof direct manipulation. Then, instead of playing, perform-ers ”control” their instruments. John Fulleman [6], a fre-quent collaborator of John Cage, attributes David Tudor an”ability to assert just enough control over the equipmentto get through a concert”.

In a lecture-talk given at the Oxford University [3] JamesMooney explains how within an interview to the Englishexperimental music band Gentle Fire in 1970, the bandmember Richard Bernas describes how he plays a customsensor-based electronic music instrument called qHong.Bernas assures that: ”the instrument is the score of whatwe are playing”. On his talk called ”the instrument is thescore” Mooney develops a framework where the relation-ships between instrument and score can be defined throughshaping the affordances the instrument creates. For beingmore exact, its range of affordances. The concept of affor-dance in musical instruments will be explained and con-textualized later in this paper. In addition, Mooney rec-ognizes performers as another active shaping element ofthe musical composition. Consequently, for Mooney per-formers would have a crucial role in defining the musicalwork. Later this paper will recover Mooney’s observa-tions for proposing a theoretical framework for designinginstrument-scores.

1 Many artists labeled as postmodernists i.e. Frank Zappa or John Zorndeclared how they were deeply influenced by this musical practice

2. PERFORMATIVE MATERIALITY

An important conclusion from our previous section is thatinherent scores would be the result of an instrumental prac-tice. Inherent scores only exist in virtue of a performer’scommitment on interpreting some type of materiality asperformative, being of physical, virtual or mixed origin. Inorder to explain the foundations of this instrumental prac-tice, we will make use of the theory of performative mate-riality.

Jacucci and Wagner [7] have explained why the materi-ality of electronic musical instruments is not only a meresupport for acoustic or digital sound machines. This ma-teriality is performative too: ”material artefacts have ahistory, emerge as part of specific events in time and be-come part of performative action”. Physical materialityhas always a performative potential. The theory and ap-plication of performative materiality within Human Com-puter Interaction has been extensively studied by JohannaDrucker [8]. Drucker suggests that the materiality of asystem ”only occurs when we action it, and only and atthat moment we perceive and discover it, always distinctin each instance”. For Drucker, ”material conditions pro-vide an inscriptional base, a score, a point of departure, aprovocation, from which a work is produced as an event”.Certainly, as Brown and Duguid [27] have emphasized,material features, in their peripheral, evocative, and ref-erential function, provide border resources for interaction.But can these features be considered scores?

In contemporary performative arts, scores can take di-verse forms and materials: graphic scores, action scores,computational, sculptural, etc. Then, can anything be ascore? The choreographer William Forsythe pointed out[9] that a score ”represents the potential of perceptual phe-nomena to instigate action, the result of which can be per-ceived by a sense of a different order”. Following this idea,in traditional Western music a score would be the instiga-tor of a transition from the visual to the aural via our body.In the case of non-traditional notation, how does a scoredefine the way we interpret a musical work? Forsytheexplains that ”a score is by nature open to a full paletteof phenomenological instigations because it acknowledgesthe body as wholly designed to persistently read every sig-nal from its environment”. Forsythe lays more importanceon the embodied relation with the performer: scores ap-pear in the exact moment when a performer finds a per-formative potential within an object or a concept, decid-ing its phenomenological outcome on an open basis. Thus,each object would be an embodied device open to the phe-nomenological interpretation: a potential available to itsconversion into a performative event. We could concludethat under this vision, any object has the potential to be ascore. And definitely our musical instruments incorporatethis potential too. Finally, if musical instruments are per-ceived as scores that would be essentially in virtue of theirphysical or performative materiality, as well as their highevocative power to instigate musical actions.

Under this generalized definition of score, it is now pos-sible to interpret electronic musical instruments as scores.Not mainly because of being musical instruments, but es-

sentially from the performative materiality they engage.The generalization of this fact would suggest that any ob-ject can be a score. For exemplifying this idea, JamesMooney [6] attributes the creation of improvisations out offound objects to composer Hugh Davies, who ”re-purposed(objects) as musical scores”, in a way that ”any visualstimulus can be interpreted as a set of instructions thatshape the development of music”.

Additionally, we should clarify that physical material-ity is not the only in charge of shaping the musical work.Nowadays, a great part of our digital instruments base theirfunctionalities on the control of graphical user interfaces(GUIs). In this case, their performative materiality can-not be expressed through physical artefacts. From ThorMagnusson’s research [10] we understand better how theinherent affordances and constraints of the constituent ele-ments of graphical interfaces mediate on screen-based mu-sical instruments. In a certain way, GUIs and tangibles canbe unified by the theory of performative materiality.

3. NOTATION

3.1 Preliminary Questions

The idea that an instrument can be a score affords creativerelationships. But, it also carries very substantial ques-tions. For example, if an instrument is a score, is it true thata score is an instrument too? If one thing can be the otherat the same time, are both the same thing? If an instru-ment is a score, can one part be separated from the other?Where can we physically find this inherent score withinthe instrument’s body? Is an inherent score the additionof both forms or is it a new synthetic thing? How is aninstrument-score interpreted or shaped by its performers?

For fully understanding the nature of inherent scores, theyhave to be contextualized within the ontology of notation,arts and music.

3.2 Notational Systems

Along this section, we are adopting Nelson Goodman’s no-tation principles taken from Languages of Art [11] in orderto spell out the kind of notation behind inherent scores.

If inherent scores are scores, it is because they manifestsymbols to their readers. A symbol in a notational sys-tem refers to something (literal, metaphorical, indirect) andits interpretation depends on the system of symbolization.Furthermore, the sort of symbol it is -linguistic, musical,pictorial, diagrammatic, etc.- will be in virtue of its be-longing to a specific system.

A symbol system, say, the English language, actually con-sists of a symbol scheme -i.e., of a collection of characters-with rules to combine them into new, compound charactersassociated to a field of reference. For Goodman, symbolsystems are notational when:

1) the characters are correlated to the field of referenceunambiguously (with no character being correlated to morethan one class of reference, or compliance class)

2) what a character refers to -the compliance class- mustnot intersect the compliance class of another character (i.e.,the characters must be semantically disjoint)

3) it is always possible to determine to which symbolan item in the field of reference complies (i.e., the systemmust be, semantically, finitely differentiated).

Languages like English have a notational scheme but failto be a notational system because of ambiguities (in En-glish, cape refers to a piece of land as well as to a piece ofclothing) and lack of semantic disjointness (man and doc-tor have some referents in common).

Finally, let’s apply these definitions to our artistic field.Sculptural or pictorial systems fail on both syntactic andsemantic grounds so they are non-notational systems. WithinGoodman’s approach, a musical score is a character in anotational system only if it determines which performancesbelong to the work and, at the same time, is determinedby each of those performances. Given the notational sys-tem and a performance of a score, the score is recoverable.This is ensured by the fact, and only by the fact, that thelanguage in which a score is written must be notational, soit must satisfy Goodman’s stated requirements.

3.3 Inherent Scores Notational Scheme

For bearing out the notational scheme of an inherent scorewe need to examine its symbol system and rules. If mu-sicians consider that instruments, through their constraintsand affordances, are scores too, then some kind of sym-bols and rules must exist. In this section we will first clar-ify where these symbols can be found. This analysis willhelp us to conclude if this system is notational or not. Itis important to remark that our task here is not showingif it is possible to create a Goodman’s notational systemfor electronic instruments. Our focus is on understandingwhat kind of notation is the one of an instrument, or thenotation of an inherent score.

The initial and probably main complication consists inthe total absence of rules within the field of reference. Nor-mally, within instruments materiality i.e. a cello, elementsare not discretized. The space of affordances against ma-teriality is continuous. For this reason, traditional West-ern musical notation establishes a radical discretization onthis space of affordances. For example, in Western mu-sical notation, the space of frequencies has been entirelydiscretized with the use of notes and scales. Or as anotherexample, among all the possible sounds that i.e. a cellocan produce, our traditional Western notation has filteredout all kind of noises, being centered on the production oftones.

Additionally, within this continuum of materiality, if aconstituent is defined as a symbol or not, it is a decisionleft to the actual performer of the inherent score. Goodmanexplains that these kind of systems are essentially analogsystems. For every character there is an infinite amountof others such that referring to the same mark. We cannotpossibly determine that the mark does not belong to all andsuch that for some object we cannot possibly determinethat the object does not comply with all. A system like thiskind is obviously the very antithesis of a notational system.

Figure 2. Earle Brown’s December 1952

3.4 Non-Notational Systems and Musical Graphs

In the search for defining the properties of an inherent scoreand its symbols, there is a clear lack of terminology to ap-ply here. For this purpose, we propose first analyzing thenotational scheme of graphic scores, which nowadays arean accepted format of scores while they have been exten-sively studied within the ontology of music. From theseresults it would be easier to extrapolate some parts of ouranalysis. Although graphic and inherent scores are notthe same thing they share many instrumental similarities.Later in this paper we will explain some interesting differ-ences applied to instrumentality. Nevertheless, their termi-nology can be used for incorporating our inherent scoreswithin the ontology of music.

Graphic scores appeared in the musical avant-garde asa way to release composers from the constraints of writ-ing their music using the notation of a traditional Westernscore. Consequently the representation of a musical ideaopened to the personal and subjective selection of graphicfigures that inspire new and imaginative ways of interpre-tation. One of the first examples of graphic scores is EarleBrown’ December 1952 (Figure 2).

Are graphic scores notational systems? Earle Brown didnot specify how his graphical symbols should be inter-preted. Therefore, depending on just how the symbolsare interpreted, syntactic and semantic disjointness may belacking. In cases like December 1952, composers are usingsystems that only slightly restrict the performer’s freedomto play what and as he pleases. The system furnishes nomeans of identifying a work from performance to perfor-mance. Furthermore, we can say that the system of De-cember 1952 is non-notational, like inherent scores.

An early but fundamental contribution describing, illus-trating and classifying the symbols used by modern com-posers was Erhard Karkoschka’s Notation in New Music(1965) [12]. Karkoschka developed the following typol-ogy of musical systems:

- Precise Notation: where every note is named- Range Notation: where for example, only the limits of

the ranges of notes are set- Suggestive Notation: where at most relations of notes,

or approximate limits of ranges, are specified.- Musical GraphicsCertainly, musical graphics are non-notational because

they lack both syntactic or semantic articulation. We shouldnote Karkoschka’s intuition in not calling them musicalgraphics notation. In Languages of Art Goodman explainsthat Musical Graphics are another example of analog sys-tems. For every character there is an infinite amount ofothers referring to the same mark.

We must remark that musical graphics -as they were coinedby Karkoschka- are non-notational systems but they arestill scores. The implications of classifying traditional West-ern scores as notational systems and musical graphs asnon-notational do not restrict us from saying that Decem-ber 1952 is a score. The appreciation of -what is- and -whatis not- a score has changed historically with the introduc-tion of new musical poetics. Finally, it is a fact that EarleBrown’s December 1952 has inspired hundreds of musicalrealizations. Thus it must be a score.

A very important property of graphic scores is that theyare usually not created with the interest of substituting a”normal” score. As discussed by Rebelo [13] or Vlagopou-los[14], and by Earle Brown himself on his seminal OnDecember 1952, graphic scores are usually created as im-provisational scores. They appear with the mere intentionof guaranteeing an unique way of performing. But graphicscores are not ”the performance”. Furthermore, the graphicscore is a trigger for the interrelation among performers inthe rehearsal phase, if it exists. A graphic score is a provo-cation to solve a musical challenge with our own poeticson communication with ourselves and the other perform-ers. This strategy would be congruent with the practice ofmusical improvisation. For suggesting an open improvisa-tion, certainly a non-notational system can be a very validcreative trigger. Any effort in the direction of discretiz-ing the system of symbols used during a performance (e.g.with a notational system) would lead to discretization ofthe musical response as well.

Updating Karkoschka’s typology, nowadays our musicalgraphs can adopt any form and any dimension. It is re-markable that historically a big percentage of these graphicscores have eminently used paper as the medium, in aninexplicable and non necessary conceptual analogy to thetraditional format of the traditional score. We had to waituntil the advent of digital interfaces to see musical nota-tions that can be interactive, dynamic, fragmented or nonlinear. Examples would include the animated scores ofMiyashita [15] or the three-dimensional scores of Berghaus[16].

3.5 Instrumentality of Graphic and Inherent Scores

Graphic and inherent scores are non-notational systems man-ifesting interesting differences in their instrumentality.

The first difference we can observe is that graphic scoresexist in physical or virtual forms. They can adopt diverseforms. Even they can be virtual or dynamic. But theyare always perceived through our senses and they can be

analyzed before we start playing the instrument. On thecontrary, inherent scores display themselves through thecreative exploration of the constituents of a musical in-strument. And this happens only at the moment of per-formance. Therefore, inherent scores cannot be formallyunderstood as objects. They are a purely mental activityenacted from the interaction of a performer with an instru-ment.

Interestingly, when an author or a performer declares thata visual composition should be considered as a graphicscore, the original graphic object extends itself towardsmusic and in a non physical way. It acquires a new ab-stract dimension able to enact musical compositions. Thegraphic object becomes a new mental category, in a sim-ilar way to inherent scores. However, conceiving musicfrom a graphic score demands some rational. Performinga graphic score means giving visually perceived contenta musical meaning. As we remarked, graphic scores donot substitute traditional scores. They are a kind of men-tal provocation. But this translation from the visual to theaural needs an interpreter, being it an individual, a collec-tive or even a technological device. To this rational manyexternal elements can be added: in-situ possibilities, so-ciocultural influences, etc. All this plethora of informationmakes the realization of a graphic score an unique musicalwork, intimately connected to its performers.

On the contrary, there is nothing to translate when per-forming inherent scores. The decision-making process dur-ing performance is normally intimately connected with ourcreative exploration and the resulting sonic reinforcements.Understanding the specific performative potentials of aninstrument is an a posteriori process. It happens once wehave already started playing. The performer of an inherentscore does not need a translation from the physical to theaural. Even more, many times the instrument can soundwithout our interaction 2 . Therefore, the performer’s taskis closer to the role of controlling or modifying this con-tinuous flow of sound. Indeed, many times, these sonic af-fordances are not predictable. They can change or evolveduring a performance with the conditions of the room, thesituation of the performer or the instrument configuration.All this makes very difficult to prepare a concert plan in ad-vance but allows ample space for experimentation. In con-trast, for playing graphic scores, performers usually willrequire some aprioristic thinking. Sounds usually are pro-duced after some cognitive process of interpretation fromthe graphic elements found in the score.

4. THE FORM OF AN INHERENT SCORE

4.1 Hybrid Arts Forms

Another substantial problem is the artistic form of the mu-sical work that an inherent score affords. And how it func-tions in relation to other visual or physical elements exist-ing in the instrument.

Acoustic instruments are eminently defined from theirphysical materiality. Electronic music instruments consist

2 For example a ”voltage controlled oscillator” can sound since themoment it is connected to a power supply

of hardware and software. Both the visual and physicalpart of an instrument can be specifically designed to infersome kind of limitation or, in other words, to shape the mu-sical work. Inherent scores are the combination of existingforms resulting in a kind of hybridization.

Within the ontology of arts, philosophers have studied theidentity or nature of the art object in physical arts (paint-ing, sculpture, etc) and in the so called non-physical arts(mainly music and literature). In the latter, there is no par-ticular ”thing” to be considered the artwork itself. Thescore of a musical sonata or the printed paper of a novelare not considered the art object itself but its representa-tion. Some authors like Croce [17], have suggested thatmusic and literary works are purely mental.

In philosophy, there is a wide-spread consensus 3 that amusical work is a variety of abstract object, a structuraltype or kind. If like in classical Western music, composershave not created the sounds to be heard in a performance,where is the actual work? Is it in the score? Scores arethe mere representation of the musical work. They are thesymbols to concatenate during a performance plan. Butscores do not sound per definition so they cannot be themusical work. Introducing the wide debate on the identityof the musical work would require a longer extension butsince there are artists claiming that their interfaces and in-struments are scores, then we should at least introduce theproblem of defining the musical object into our discussion.

An interesting approach for deducing the form of inher-ent scores can be taken from Jerrol Levinson’s theory ofhybrid art [18]. Levinson notes that not all kind of artsare pure, some are hybrids. Examples like kinetic sculp-ture or interactive audiovisual installation show us that in-dependently of their complexity, these forms of art showelements of multiple art forms. A kinetic sculpture wouldbe the encounter of sculpture and dance. An audiovisualinstallation would consist of multiple media: cinema, mu-sic, sculpture, etc. On the contrary, we perceive a tradi-tional figurative painting as an instance of its category. ForLevinson the hybrid status is primarily a historical thing,in a way, as is being a biological hybrid. An art form is hy-brid ”only in virtue of its development and origin, in virtueof its emergence out of a field of previously existing artisticactivities and concerns, two or more of which it in somesense combines”. Inherent scores are good candidates tobe considered hybrid forms. At this stage of the explana-tion, we could describe them intuitively as the mixture ofa musical work and performative materiality enacted fromphysical or virtual objects. For us, the most important fea-ture that this theory exhibits is that if an art form is hybridthen it must be understood in terms of the combination ofits original components.

Levinson extends his theory of hybrid art to the combi-nation of existing art forms and technological processes.For example, laser sculpture, computer music. computergraphics, video installation, etc would be a result of this

3 A complete review on the debate of ”what a musical work is” cannotbe afforded here, but we can divide between two actual tendencies. FirstPlatonist or realist theorists holding that musical works are collectionsof concrete particulars e.g. Goodman 1968, Kivy 1983, Levinson 1980,Davies 2001; and those anti-realists who deny there are any such thing asmusical works e.g. Rudner 1950, Cameron 2008, Stecker 2009.

combination. Thus, Levinson features clearly the plausi-bility of new art species creation from the hybridizationwith technologies. The resulting possibilities for this pro-cess are three: juxtaposition (or addition), synthesis (or fu-sion) and transformation (or alteration). In all these threecases of process, Levinson explains that the hybrid com-bination of art form A and B to produce C, will changethe properties that A or B exemplifies in the joint context.These properties would be relative to what one of the origi-nal forms would exemplify on its own, or at least affect theprominence of what each exemplifies after combination.

As we have discussed before, the object embedding aninherent score stays in an identical physical form after itsperception as a score too. The consideration that an instru-ment is a score is produced at a mental level. Thus, thischange of perception does not carry physical changes. Thesame occurs in the case of graphic scores. Graphic scoresseem to be a good example of hybrid art form resultingfrom painting and music. As well in this case, the consider-ation that ”a painting as a score” does not bring changes tothe painting form itself. In graphic scores, this hybridiza-tion changes the perception of a very known physical ob-ject (the painting) transforming it into a hybrid of a physi-cal and a non-physical entity, an object and a musical work.The same would happen to a sculpture if at some momentwe manifest understanding it as a score. This example sup-ports the idea that hybrid art forms are essentially new his-torical forms. Once the artistic practice adopts a hybrid artform and it becomes general, we will not refer to its hybridorigin anymore.

Coming back to the discussion on our inherent scores,we have gained an adequate theoretical framework for in-terpreting the combination of both the physical and perfor-mative materiality of an instrument as a new type of hy-brid form. This hybrid art form, the inherent score, wouldbe the fusion of two existing forms (physical materialityand performative materiality) resulting on the synthesis ofa new kind. Like in the case of graphic scores, the exis-tence of this new hybrid form is congruent with the per-ception of performers of being playing a score when theymanipulate the instrument. Performers have the perceptionof playing a specific form. Therefore, this inherent scorewould be a new abstract object perceived no longer as onlythe physical instrument. It is perceived as a performativepotential of the instrument shaping every moment duringthe act of playing. Certainly, the result of this fusion altersthe perception of the original forms. Instrument’s physi-cal materiality gets augmented and extends itself towardsa compositional object, acquiring some abstract attribute.In the same way, performative materiality gets some kindof order. It defines itself for a specific use and a particularperformer.

At this stage we can now answer some of our preliminaryquestions we left open. For example, when an instrumentis a score, our question was if one part can be separatedfrom the other. Having concluded that an inherent score isa new abstract object synthesized from the fusion of twoalready existing, we can now assert that this separation isnot possible. There is nothing to separate. The instrument

still exists but a new abstract musical object appears onstage as an attribute in the system. There is no possibil-ity of explaining the inherent score to others without theinstrument itself. Additionally, as we have shown beforein this paper, this inherent score is eminently a subjectiveand mental creative attribute that can be interpreted differ-ently by every performer. Separating the score from theinstrument if possible, would still require information onthe performer involved to be understood. The importanceof performers for shaping an inherent score will be moreexhaustively analyzed in this paper when studying theirphysical embodiment (section 5.2).

Another question formulated was if in the case an instru-ment is considered as a score, then if it is true that a scorewould be an instrument too. As we have explained before,every object can have specific performative potentials tobe perceived as a score. On the contrary, inherent scoresare abstract musical objects and they usually do not embedany specialized feature for music creation except the nat-ural sound their physical constituents produce. Therefore,in general scores are not perceived as instruments 4 .

4.2 Inherent Scores: typology of symbols

For the analysis of this hypothesis of hybridization we aresuggesting here a typology of symbols that we can findwithin our instruments. Due to the important role that tech-nology holds at many of our actual music instruments, wewill focus our attention on those instruments incorporatingsome kind of computational system behind their configu-ration.

The first type of symbols are purely extrinsic. Thesewould be mainly representational. They give us informa-tion about the computational status of the musical instru-ment being played. For example, the visual composition oftokens on a table-top interface like a Reactable 5 [19] (Fig-ure 3) is an example of extrinsic symbols. They representthe status of the algorithms an user is running at every mo-ment and how they are interconnected. In this case, thesystems affords less on the materiality of these tokens (e.g.form, color, material, texture, etc). If instead of using theseoriginal acrylic tokens we use other ones made of wood,the sound mapping or the overall sonic output will not beaffected.

The second type of symbols are intrinsic. These would beinherent to the affordances of the physical interface. Fur-thermore, we talk about intrinsic symbols when their phys-ical affordances determine various features of what is au-rally enacted. Using the same example, Reactable’s roundtable form affords to the multi-player or collaborative per-formance. Through this specific intrinsic property, the in-strument’s materiality definitely shapes performative ma-teriality and finally the way the instrument can be played.

4 An exceptional example of ”a score that sounds” is the project Tan-gible Scores, a technological hybrid allowing a visual score being thecontroller of a sound synthesis. This will be showcased later in this paperat section 6.1

5 The Reactable is a round form electronic music instrument. By plac-ing blocks called tangibles on the table, and interfacing with the visualdisplay via the tangibles or fingertips, a virtual modular synthesizer isoperated, creating music or sound effects.

Figure 3. A Reactable. Photo by Daniel Williams

We can find both types of symbols within instruments andtheir combinations are possible. For example, a modifiedReactable incorporating the same type of representationaltokens but featuring recognition of specific physical prop-erties of these tokens. For example, the stretch force ap-plied to rubber made tokens could control the volume ofan associated sound synthesis. In this case the extrinsiccomposition of tokens can be affected by the properties ofthe intrinsic physical materiality supporting it.

5. COMPOSING INHERENT SCORES

5.1 Affordances and Constraints of an instrument

For composing the inherent score every instrument embedswe will follow the principles proposed by James Mooney[3]: shaping its affordances and constraints.

Affordances, as psychologist J.J.Gibson defined them, arethe properties of the relationship between the environmentand the agent. In our case, the environment would be themusical instrument as a reference frame. The agent wouldbe a performer. Between agent and environment, infiniterelationships can be created, but the potentials of perform-ing some event are less probable than others. Sometimeseven impossible. A violin affords playing sounds, but itdoes not afford traveling.

A remarkable propery of affordances is that they are highlydependent on the reference frame where they are inscribed.For example, cultural contexts or personal backgrounds.What an object affords to a a person can be different toanother, even living in the same sociocultural environment[20]. Therefore, affordances could be essentially subjec-tive perceptions influenced by our social constructs. Andthis condition can reach the maximum of dependency inthe case of performative arts. From classic ethnographicstudies we know how performances are central to humanunderstanding [21] and post-modernism have drawn atten-tion to the way performances seek to reinforce and com-municate our identities in society [22]. Recent researchon socio-situated interface design [23] is coherent with theidea of socio-oriented performing frames. These theoriessuggest that when using an interface, cognitive scaffoldscan only exist in the context of a social setting. Certainly,

the capability of performing carries a substantial contextand the sociological ecology of acting in front of others.

In parallel we have the notion of constraint. The appli-cation of the concepts of affordance and constraint in elec-tronic music instruments has been deeply studied by ThorMagnusson. The author explains [10] how Margaret A.Boden [25] defines constraints as one of the fundamentalsources for creativity: [F]ar from being the antithesis ofcreativity, constraints on thinking are what make it possi-ble. ...Constraints map out a territory of structural pos-sibilities which can then be explored, and perhaps trans-formed to give another one.. This assessment remarks thepotential of constraints as a trigger for creative enactments.Constraints would be characters in the performative mate-riality of an object, being it physical or virtual.

Within the discipline of improvisation with electronics,the instrument’s affordances take the important role of shap-ing the way an interface is played through its different con-straints. But more important for our discussion, if theyare considered as scores then they would suggest what isplayed too. As we have seen before in this paper, this inher-ent score can afford interesting performative enactments.

Mooney [3] supports the idea that a musical instrumentcan be designed from the perspective of which kind ofmusic relationships it affords. Also, Mooney identifiesthe possibility of defining the ”spectrum of musical affor-dance” of instruments. This can be achieved by designingthe instrument or interface and establishing a range of mu-sical practices the instrument can support. For example,although very complex textures of sounds can be playedand controlled with a Reactable, it would be rather diffi-cult to play Mary had a little lamb. It is then noticeablethat the spectrum of affordance is not comparable to diffi-culty or to complexity of the instrument. Affordances arefully mediated by the embodied relationship between in-strument and performer. Even if the performer knows themusical notes of Mary had a little lamb it will be impos-sible to play it correctly on time with a Reactable. There-fore, instrument’s performative affordance and other typesof affordance, like musical affordance, expressiveness af-fordance, etc cannot necessarily match.

5.2 Physical Embodiment

As we have seen, performing with an electronic music in-strument would be the creation of relations and meaning-ful structures between the inherent score and its enactedaudiovisual interpretations. But these are always mediatedand shaped by our embodiment. A good instrumentalist isable to create embodied relationships with the instrument,leading to a feeling of intimacy and control. Undoubtedlythis will be perceived as a key factor to evaluate the expres-siveness of a good performance.

This evidence was used by Mooney [3] to introduce intu-itively the ”performer” as another shaping parameter of themusical work. If the instrument is the score, then many ofthe decisions taken, even shaped by the instrument, will beresult of performer’s acts in freedom. Although Mooneydid not develop further this argument, he introduced an-other variable in the equation: performer’s reference frame

Figure 4. A Tangible Score example.

and performer’s embodiment. First, the sociocultural con-text of the performer, even the actual mental conditions atthe moment of approaching a performance will shape itsresult. For example, the expressiveness of a first musicalapproach with a Reactable depends highly on knowing theelements of computer music in advance (what is a synthe-sizer, a sequencer, etc). Second, more objective factorsconnected with the embodiment can conduct the musicaloutcome. For example, if the electronic instrument de-pends highly on a physical ability that cannot be achievedby a specific performer i.e. through some disability, allthe performative affordances designed can appear hiddenor invisible.

Thus, we can conclude that design models centered onlyon defining constraints and affordances must include ”theperformer” as an influencial parameter. Therefore, we couldonly speak of inherent scores when connected to a partic-ular performer. In Rainforest we would describe Tudor’sversion, John Cage’s realization, etc. Probably all the in-stances of the same musical work will be very different.

6. TWO EXAMPLES

6.1 Tangible Scores

In 2014, together with Martin Kaltenbrunner, I presentedthe paper Tangible Scores: Shaping the Instrument Inher-ent Score [1] at the New Interfaces for Musical Expressionconference (NIME). Tangible Scores are a new paradigmfor musical instrument design with a physical configura-tion inspired by graphic scores (figure 4). This instrumentimplements practically many of the concepts and ideas ofthe so called instrument-scores, and it has been reviewedby Maestri and Vlagopoulos a the first TENOR conferencein 2015 [2]. Many aspects of the theories explained herewere achieved during the practical development of Tangi-bles Scores. Therefore, my intention is now contextualiz-ing the instrument Tangible Scores within the frameworkpreviously explained.

A Tangible Score 6 is a tactile interface for musical ex-pression that incorporates a score in its physical shape,

6 Full information on the project and videos can be found online at thefollowing URL: http://interface.ufg.ac.at/tmg/projects/tangible-scores/

surface structure or spatial configuration. Creating an intu-itive, modular and expressive instrument for textural mu-sic was the primary driving force. Following these criteria,we literally incorporated a musical score onto the surfaceof the instrument as a way of continuously controlling sev-eral parameters of the sound synthesis. Tangible Scores areplayed with both hands and they can adopt multiple phys-ical forms. Complex and expressive sound textures can beeasily played over a variety of timbres, enabling precisecontrol in a natural manner. Using sound as a continuousinput signal, both synthesis and control are available si-multaneously through direct manipulation on the engravedpatterns of the physical score.

Every tangible score is conceived from a different graph-ical score (Figure 5) that still represents a musical idea butit has been also specially designed for providing a diversepalette of acoustic signals when touched. But more impor-tant, the tactile scores define and propose specific gesturalbehaviors due to the different affordances and constraintsof the object in front. Sound is generated through a poly-phonic concatenative synthesis driven by a real-time anal-ysis and classification of input signal spectra. Each of thescores is loaded with a specific sound corpus that definesits sonic identity. Thus, a tangible score provides a im-plicit visual and haptic feedback in addition to its soniccore functionality, making it intuitive and learnable but aswell suitable as an interface for musical improvisation andsonic exploration.

At the moment of designing our paradigm, we were quiteinfluenced by Lucier’s quote: ”there were no scores to fol-low; the scores were inherent in the circuitry”. We ac-cepted it. It matched our instrumentalist intuition as longtime electroacoustic music improvisers. Additionally, itwas possible for us to contextualize the instrument withinthe field of tangible interfaces and human computer inter-action (where we have been working for a long time too).

We first understood that, not only musical instruments,but any physical or virtual object loaded with performa-tive materiality affords being played. And second, thatany physical or virtual object has the potential of insti-gate actions so it can potentially afford being interpretedas a score. Then, our direction had to follow the directionof trying to shape those potentials. By intentionally limit-ing or constraining the infinite possible interpretations of aspecific object within its reference frame, we are shapingthe inherent score it contains. And we do it in a deliberateact of musical composition.

In order to shape those potentials, we decided conductingor inspiring particular gestures by incorporating contrast-ing visual and tangible patterns on the surface of the instru-ment. For the first series of tangible scores, we first com-posed a set of generative patterns that we later engravedon wooden surfaces. For finding the adequate patterns, notonly an attractive a visual or gestural idea was searched.We had to negotiate its form with the adequate sonic re-sult when touched. This relationship mediates radicallythe sound synthesis of the instrument.

For evaluating the instrument, apart from performing withthem at different concerts, we offered and showcased some

Figure 5. Different designs for Tangible Scores

tangible scores to professional performers and composers(mainly in Linz, Austria). Additionally, we showcased theinstrument as a sound installation during two mass audi-ence festivals (Sonar and Ars Electronica 2014) were thou-sands of visitors could play it. From the analysis of theseexperiences, for us was clearly evaluated and proved thatboth physical gestures and sound gestures were mainly in-spired by the visual and tangible patterns: their direction,size, intention, etc.

One important decision taken in this first series of tangi-ble scores was that our design should rely only on intrinsicelements or symbols. Although a tangible scores is fullya digital instrument, we decided not displaying representa-tional information on the instrument. Due to this decision,the computational status is hidden and the sonic mappingdepends intimately on the embodied relationship betweenplayer and instrument. Due to this unification of score andinstrument, the instrument provides the representation andcontrol within a single musical artifact, fully concentratingthe performer’s attention on the interaction with the musi-cal composition in a physical way.

6.2 Choreographic Objects

Within the field of contemporary dance, the choreographerWilliam Forsythe created the concept of choreographic ob-jects [9]. Physical objects, of various types, are consideredchoreographic when they are able to enact particular be-haviors and movements via ballet dancers. These objectsreveal a choreography that is inherent to their physical ma-teriality.

This idea comes from Forsythe’s intuition on perceivingevery object as a source of enactments. As we describedearlier in this paper, for Forsythe a score ”represents thepotential of perceptual phenomena to instigate action, theresult of which can be perceived by a sense of a differentorder”. As well ”a score is by nature open to a full paletteof phenomenological instigations because it acknowledgesthe body as wholly designed to persistently read every sig-nal from its environment”.

An example of the use of choreographic objects is thework Nowhere and Everywhere at the Same Time, No.2(Figure 6) created for a solo dancer and 400 pendulumssuspended from automatic grids. When activated they ini-

Figure 6. The choreographic object ”Nowhere and Every-where at the Same Time, No.2” by W. Forsythe.

tiate a sweeping 15 part counterpoint of tempi, spacial jux-taposition and gradients of centrifugal force which offersa constantly morphing labyrinth of significant complex-ity. This setup privileges the unconscious choreographiccompetence induced by this special choreographic situa-tion. For Erin Manning [24], choreographic objects are”an affordance that provokes a singular taking-form: theconjunctive force for the activity of relation”.

In a similar way to tangible scores, choreographic objectshave been traditionally considered a constraint. Forsythedevelops an active stage, a composed reference frame suit-able for performative enactments. If stage scenographiesare usually representational, choreographic objects do notrepresent anything else than a potential to move.

In the case of the work Nowhere and Everywhere at theSame Time, No.2, Forsythe gives a fundamental form to thespace. Through this morphophoric affordance, the dance-able space and all the possible movements are discretizedby the inertial materiality of the pendulums. In this work,the dancing score can be found in the performative affor-dance of a myriad pendulums defining a composed spacearound them.

Undoubtedly, choreographic objects have the ability ofinducting creative movements and gestures in its perform-ers. Especially in large spaces and multi-user contexts, weare convinced that the notion of choreographic objects canbe useful to inspire the creation of novel interpretation ofscores as well as new phenomenological enactments.

7. CONCLUSIONS

Along this paper, we have proved the validity of using theconcept of inherent scores for describing the mediated rela-tionship between performer, score and instruments. Espe-cially in electronic music instruments. We have explainedhow the theory of performative materiality serves well toexplain the fact that any object can be understood as ascore. We have defined the notational scheme of inher-ent scores as non-notational and we have described the re-markable differences in instrumentality between inherentand graphic scores. We have elucidated the nature of inher-ent scores particularizing them as a hybrid forms resulting

from the fusion of performative and physical materiality.Finally, we have proposed a framework for composing orshaping musical works, demonstrating its possibilities withtwo examples: tangible scores and choreographic objects.

8. REFERENCES

[1] E. Tomas, M. Kaltenbrunner, Tangible Scores: Shap-ing the Inherent Instrument Score, Proceedings of theInternational Conference for New Interfaces for Musi-cal Expression, 2014

[2] E. Maestri, P- Antoniadis, ”Notation as Instrument:From Representation to Enaction”, First InternationalConference on Technologies for Music Notation andRepresentation, 2015

[3] J. Mooney, ”The instrument is the Score”: Relation-ships between Instrument, Performer and Score. Sym-posium for Performance of Electronic and Experimen-tal Music (SPEEC) Oxford University, 2012

[4] A. Lucier, ”Origins of a Form: Acoustic Exploration,Science and Incessancy”. Leonardo Music Journal 8,pp. 5-11, 1998.

[5] R. Schnell, M. Battier, ”Introducing composed instru-ments, technical and musicological implications”, Pro-ceedings of the 2002 conference on New interfaces formusical expression, 2002

[6] R. Kuivila, ”Open Sources: Words, Circuits, and theNotation/Realization Relation in the Music of DavidTudor”, Presented at the Getty Research Institute Sym-posium, ”The Art of David Tudor”, 2001

[7] G. Jacucci, I. Wagner, ”Performative Roles of Materi-ality for Collective Creativity”, Proceedings of the 6thACM SIGCHI conference on Creativity and cognition,2007

[8] J. Drucker, ”Performative Materiality and TheoreticalApproaches to Interface,” DHQ, forthcoming Winter,2012

[9] W. Forsythe, ”Choreographic Objects”, Internet Essay:http://www.williamforsythe.de/essay.html

[10] T. Magnusson (2010) Designing constraints: compos-ing and performing with digital musical systems Com-puter Music Journal, 34 (4). pp. 62-73, 2010.

[11] N. Goodman, ”Languages of Art”, 1968.

[12] E. Karkoschka, ”Notation in New Music”, 1965.

[13] P. Rebelo, ”Composing with Graphics: Revealing theCompositional Process through Performance”, First In-ternational Conference on Technologies for Music No-tation and Representation (TENOR) , 2015.

[14] P. Vlagopoulos, ”Is It All Autographic? Samples fromthe Musical Avant-Garde of the 60s”, 12th Interna-tional Conference on Music Perception and Cognition(ICMPC), 2012.

[15] H. Miyashita. Graphic Score Animation for Two Pi-anos, Feb. 28th, 2003. Toyama shimin plaza, Japan.

[16] M. Berghaus, ”Some Ideas for Three-Dimensional Mu-sical Scores”, Leonardo Music Journal Volume 21, pp.7-8, 2011

[17] B. Croce, ”The Aesthetic as the Science of Expressionand of the Linguistic in General”, Cambridge: Cam-bridge University Press, 1992.

[18] J. Levinson, ”What a Musical Work Is, Again”, in Mu-sic, Art, and Metaphysics, Ithaca, NY: Cornell Univer-sity Press, 215263, 1990.

[19] S. Jorda, M. Kaltenbrunner, G. Geiger, R. Bencina,”The reacTable”, Proceedings of the InternationalComputer Music Conference, 2005.

[20] D. A. Norman, Affordances, Conventions, and Design.Interactions 6(3), 1999

[21] J. G. Sheridan, N. Bryan-Kinns, A. Bayliss, ”Encour-aging witting participation and performance in digitallive art”. Proceedings of the 21st British HCI GroupAnnual Conference on People and Computers, 2007

[22] G. Butler, ”Bodies that matter: on the discursive limitsof ”sex” ”. New York, London. Routledge, 1993

[23] J. Dijk, R. Lugt, C. Hummels,”Beyond distributedrepresentation: embodied cognition design supportingsocio-sensorimotor couplings”, Proceedings of the 8thInternational Conference on Tangible, Embedded andEmbodied Interaction, 2014

[24] E. Manning, ”Relationscapes: Movement, Art, Philos-ophy”. Cambridge, Mass. MIT Press, 2009

[25] A. Tanaka, Sensor -based Instruments and InteractiveMusic pp. 233-255 in R. Dean, The Oxford Handbookof Computer Music. Oxford University Press, 2009

[26] M. A. Boden, ”The Creative Mind: Myths and Mecha-nisms”. London:WiedenfieldandNicholson, 1990.

[27] J. S. Brown, P. Duguid, ”Bordeline Issues: Social andMaterial Aspects of Design”. Human-Computer Inter-action Journal 9:3-36, 1994.