Performair: music for offices

Lorenzo Cercelletta, 264883

Relatore Philip TaborCorrelatrice Gillian Crampton Smith

Sessione di laurea 31 marzo 2010

Performair:music for offices

Dorsoduro / 2206 Convento delle Terese30123 Veneziat. +39 041 257 1323f. +39 041 257 [email protected]/fda

Tesi di laureaCorso di laurea specialistica in Comunicazioni Visive e Multimediali

Performair: music for officesLorenzo CercellettaMatricola 264883a.a. 2009/2010

Relatore Philip TaborCorrelatrice Gillian Crampton Smith

Sessione di laurea 31 marzo 2010

EnglishPerformair is an audio-visual interactive installation which mis-uses traditional office devices printers, modems, scanners etc. originally designed to perform other functions, to play together as elements of an unconventional orchestra.

Performair aims to awake the everyday user from a state of cer-ebral numbness caused by the present proliferation of technologi-cal devices. What once was not only a device but also a medium, to-day appears as a simple object in an infinite network of objects. By reappraising traditional devices, new ways of auditory and visual communication can be explored.

ItalianoPerformair unistallazione audio-visiva che prende forma at-traverso un diverso utilizzo di strumenti da ufficio (stampanti, mo-dem, scanner ecc.) originariamente progettati per assolvere altre funzioni, come elementi di unorchestra non convenzionale.

Lo scopo di Performair risvegliare lutente comune dal tor-pore intellettivo causato dallattuale proliferazione di strumenti tecnologici. Quello che, una volta, non era semplicemente uno strumento ma anche un medium, oggi appare come un semplice oggetto in una rete infinita di oggetti. Attraverso una pratica di ri-considerazione circa gli strumenti da ufficio tradizionali, nuove vie di comunicazione uditiva e visuale possono essere esplorate.

Abstract

Abstract i

fig.1 previous pageParticles net (2007) - Authors ownA piece of software developed in Action-Script 2.0 which metaphorically represents the aggregation of thoughts during the ini-tial stage of this project

ii Performair: music for offices

Abstract i

1. Introduction 1

2. Research focusing and contextualisation 192.1 A case study: Lozano-Hemmers method 202.2 State of the art 242.3 Sampling: an alternative scenario 34

3. Sound and auditory perception 373.1 Sound 373.2 Auditory perception 40

4. Concept definition and methodology 434.1 Project scope: Verplanks method 44 4.2 Preliminary phase 464.3 Hardware challenge 47 4.4 Software challenge 494.5 From pre-prototyping to prototyping 52

5. Performair: final project 595.1 Why Performair? 595.2 Orchestras 60 5.3 Instruments 635.4 Sheet music and Performairs score 685.5 hui: hybrid user interface 69

6. Conclusions 73

Acknowledgements 77 Appendices 79Sources 83 List of figures 94 Notes 98

Contents iii

Contents

Introduction 1

This is a project about communication, on how communication can be shaped into design objects and how it can be established thorugh an art piece.

In The Language of New Media, Lev Manovich1 underlines the importance of text in human culture and, most important, in computer culture. He writes that text plays a primary role among media for a reason: it is, indeed, a medium like others, but also a meta-language by means of which all others media are encoded. This appears evident when concerning computer science, from the most basic to the most advanced element: the bios2 of a computer, the pixel value of images, the formatting of an html page and so forth. However, Perfomair is neither a technological nor a com-puter science project, but a design project. It is conceived from a design perspective in which text and, more specifically, typographic elements play a vital role.

In musical terms, a classical orchestra performs a piece from a score, which is nothing but a text. Indeed, a peculiar kind of text written in a particular language, with its syntax and symbols, yet a text. Performair also uses texts: the compiled text, which ena-bles devices to play as if they were musical instruments, and the printed text, as the result of the performance. The crucial difference is that, within Performair, music is not read and played, but played and then printed for the audience to read. An invisible text which throughout the performance becomes a particular kind of score. Miles Davis3 said dont play whats there, play whats not there. John Cage4 decided to play what was not there and it took him 4 minutes and 33 seconds (433).

fig.1 previous pageImpression #2 - Authors ownHow things come into focus. Picture taken before the start of the Optronica audio-visu-al festival (London, 2007)

Introduction

So as to clarify this discourse and to give a consistent theoretical background to Performair, I will now describe a series of works which tried to bridge sometimes with very effective and plain results the gap between the visual and auditory realms. It is since the invention of writing that humans explored the expressive po-tential of visual signs to share thoughts, experiences and knowl-edge. The birth of cuneiform scripts, hieroglyphics and ideograms has presumably been an answer to the human necessity of crystal-lize and, essentially, visualize things through peculiar signs or, more appropriately, graphemes. In fact, sign is quite controversial as a term to grasp. Many relevant figures approached it within different fields philosophy, of art, semantics, semiotics, psychology etc. and deeply speculated on it: Ernst H. Gombrich5, as an instance, theorised a science of signs when reviewing Charles Morriss6 Signs, Language and Behavior on the 31st issue of the Art Bulletin (Go-mbrich 1949, 68-73).

However, I would rather concentrate on how writing changed during centuries from being first predominantly illustrative to then adopt an alphabetical form. Relevant to this discourse is not the motivation probably the alphabetical system was a more versatile and easier way to structure and diffuse texts but the consequences of this event. Most of alphabetical writing languages excepted the Korean (Hangul) which is an hybrid of alphabetical and ideogram systems partially lost their inner visual component.

On the one hand we have a semi-pictorial system ideograms are visual representations whereas on the other hand a stylized system alphabets are phonetic representations. Some might have an objection to this by claiming that alphabetical letters do have a visual component. While agreeing, I would add, though, that an ideogram is more figurative than a letter. I find this a satisfactory explanation on how the design of typefaces originated. It seems also quite convincing as the reason that motivated many artists, in different times and cultures, to explore ways of illustrating with let-ters. This is what Guillaume Apollinaire7 theorised when he titled Calligrammes (figs.2,3) his collection of poems written between 1913 and 1917:

For me a calligram is an ensemble of sign, drawing and thought. It represents the shortest way to express a concept in material terms and to force the eye to accept a global vi-sion of the written word. (Barcellona 2009, 1)

In reality, this graphic art dates back to the Hellenistic era of bu-colic poets (3rd-2nd centuries BC). A few of those works survived, among them Simiass, a poet from the island of Rhodes, who shaped a series of verses as simple symbolic images the axe, egg, altar, wings to exalt mythological meaning behind his poems (figs.4,5).

Other examples can be found in the Latin as well as in the Islamic culture, where visual poetry reached astonishing peaks: in fact, the Koran prohibited the realistic representation of living be-ings and, therefore, new figurative ways were undertaken (figs.6,7).

2 Performair: music for offices

Introduction 3

figs.2,3 left, bottomWoman and Horse (1913-17) - Guillaume ApollinaireTwo examples of calligrammes. The Author uses text not only to define the contours but also to depict the clothes and body parts of figures


figs.4,5 center, bottomI. The Axe and II. The Wings (about 300 BC) - Simias of RhodesSeries of mythological poems shaped as symbolic figures

Introduction 5

figs.6,7 left, bottomIslamic calligrammes - Unknown authorRepresentation of a bird of prey and a lion in Arabic characters

Not even the technological progress i.e. the invention of typog-raphy could work as an impediment to elaborating visual com-munication through text (fig.8).

But it is only in the beginning of the twentieth century with the avant-guardes that grammatical and syntactical structures were fi-nally considered an obstacle to demolish. The poet had now to face new problems inherent to typography in technological terms: every technology has its good and bad sides as well as its rules to bypass or abide by. More than Cubists, Dadaists and Russian Constructivists, the Futurists found ingenious ways to embed mu-sical nuances within printed words. As Apollinaire said in those years, typographic tricks, pushed forward with bravery, have the advantage of originating a visual lyricism never experienced before (Barcellona 2009, 2) (figs.9,10).

Indeed, with their Tavole Parolibere (1912-1944), Futurists dis-mantled old typographic schemes and rules to evocate an onomato-poeic language made of noises and clashes. A step further stands Francesco Cangiullos8 Poesia Pentagrammata (figs.11,12): a series of typographic compositions on staff where words get translated into music. It is behind what Cangiullo defined parolibera-musicale that a more focused point of view on Performair can be outlined. The parolibera-musicale gets rid of the rigid schemes deriving from both conventional poetry and music by mixing their rules. Letters float in and out of the staff as if they were notes, vibrato and legato symbols are applied on words as if they were chords. It is in this vein that Performair can be located, taking shape as a possible so-lution to the never-ending debate between typography and music, updated to the digital era of today.

Introduction 7

figs.9,10 previous pageIl Pleut (1918) - Guillaume ApollinaireTypographical adaptation of a handwritten drawing by the author (below) representing the rain

fig.8 leftAlices Adventures In Wonderland (1865) - Lewis CarrollTypographical representation of a mouses tale


figs.11,12 bottom, following page Poesia pentagrammata (1923)Cover designed by Enrico Prampolini (be-low) and Francesco Cangiullos Piedigrotta (next page)

Introduction 9


A reference to concrete poetry closes this reflection, even though I would rather speak about visual poetry, for the term concrete is very ambiguous and objectionable. In her Concrete Poetry: A World View, Mary E. Solt9 wrote something significantly pregnant to my thesis:

Generally speaking the material of the concrete poem is lan-guage: words reduced to their elements of letters (to see) syl-lables (to hear). Some concrete poets stay with whole words. Others find fragments of letters or individual speech sounds more suited to their needs. The essential is reduced language. [...] In addition to his preoccupation with the reduction of language, the concrete poet is concerned with establishing his linguistic materials in a new relationship to space (the page or its equivalent) and/or to time (abandoning the old linear measure). Put another way this means the concrete poet is concerned with making an object to be perceived rather than read. The visual poem is intended to be seen like a painting; the sound poem is composed to be listened to like music. [...Concrete poetry], of course, asks a great deal of what used to be called the reader. He must now perceive the poem as an object and participate in the poets act of creating it, for the concrete poem communicates first and foremost its structure. (Solt 1968, i-xxi)

Perfomair begins here since its text has to be performed and then listened by an active rather than a passive listener, a translation which perfectly matches Solts view. Since we approached the ques-tion of visual language within poetry is now time to step into the realm of music from Greek meaning muse by mention-ing the question of notation - from which derives the word note. To not digress, the discourse would be reduced to minimal terms to give a simple overview on the matter.

The origin of music is not easily datable, surely it existed earlier before the invention and diffusion of the written transcription of it. The first form of music transcription is known as homophonic notation from Greek meaning same voice or sound. Only the Greeks, among ancient populations, realised a complete and advanced notation system based on the alphabet: each letter corresponded to a different tone. This system, later adopted by Ro-mans, endured till the downfall of the Western Roman Empire.

To underline is the fact that Western notation has also a great affinity with Eastern notation, this grace to the crucial role played by Byzantium. Around 9th century the most diffused system was the ekphonetic notation from Greek meaning quasi-melodic recitation of text based on grammatical accents: the acute accent indicated an ascending melody, a grave accent indi-cated a descending melody. Along with accents there were inflective marks, called neumes from the Greek meaning breathe which, developed from the gestures of the chorus conductor, in-dicated the melody development without affecting the intonation and rhythm. Later, in the Eastern Roman Empire, the neumatic no-

Introduction 11

tation became a standard mnemonic system to facilitate the reci-tation of chant. This is attested not only by the large Byzantine documentation but also by the surviving of this tradition within Greek orthodox chant. In the west, also Gregorian chant recurred to grammatical accents from Greek and Latin literature and to neu-mes, even though the neumatic system did not last long for the rise of polyphonic notation from the Greek meaning many voices or sounds based on staff (10th 17th centuries): at first neumes cohabited with the one-line staff to progressively dis-appeare as the staff lines increased to be conventionally set to five.

To precise is that notation did not change much after the 17th century. What did change was the attitude of composers towards both the staff and performers, since the notation of musical basics rhythm and sound had been solved already during the 14th cen-tury. In the period of time between the end of 19th century and the beginning of the 20th century two divergent trends appeared. On the one hand, the fracture between conductor and performer grew wider: while the performance strove to gain pre-eminence, notation became more and more deterministic and serial, this to impose the conductors will on the performers. Embryonic in this sense the cases of Schoenberg and Stravinskij.

On the other hand, together with the repudiation of the tradi-tional conception of composition, it appeared the refuse towards notation as an institutional code to abide by. The composer freed himself from the chains of traditions and conventions to finally express music in a more figurative way. The music piece was not anymore based on the sole sound component: the graphical con-figuration of the score came in first place. In fact, the musical sign progressively lost its denotative function unambiguous informa-tion requiring a determined performance to progressively acquire a connotative function ambiguous information playing on the emotive and behavioural levels. The musical sign could then be-come, to an extreme extent, purely aesthetical for it exchanged its phonic value for a visual value (figs.13,14,15).

For concrete poetry has been previously mentioned, a brief ex-cursus on concrete music is also due. A primary role within experi-mental music was surely played by Luigi Russolo10, who theorized first the abandon of harmonic sounds in favour of noise only later defined as noise music with his Larte dei rumori (1913), a manifesto dedicated to Francesco Pratella11. Not only, he also realised, with the help of Ugo Piatti12, a series of sound machines (intonarumori) and their amplifier (rumorarmonio) to produce and modulate disharmonic sounds (fig.15): this was the start of a series of musical experiments which brought into being, in the second half of the 20th century, concrete music with Pierre Schaeffer12 and electronic music.

Schaeffer, in particular, faced in his Trait des objets musicaux the problem of sound-object. The seek for new sound materials which directly derived from the need of expanding the rigid bound-aries of music: new instruments could take the place of classical instruments as the demarcation line between sound and noise was becoming thinner.

fig.13 following pageSerenata per un satellite (1969) - Bruno MadernaScore of concrete music

Introduction 13


figs.14,15 previous page, topLa Passion selon Sade (1966) - Sylvano BussottiConcrete music scores

fig.16 bottomRisveglio di una citt (1913) - Luigi RussoloMusic score for intonarumori

Introduction 15

It was the discussion started by Russolo which Schaeffer carried on by revaluating sounds within the everyday soundscape. As Solt talked about a reduced language, Schaeffer introduced the concept of reduced listening: the immediate perception of the sound itself, what he defined as sound-object, rather than the common practice of perceiving the sources associated to sounds. In a certain sense, his treatise postulated a renovation of the musical language based on the extrapolation of the self from the music context to focus exclusively on the sound matter. The connection between this thesis and Schaeffers thinking is evident. But let us get back to design.

To use an old office device the case of Performair to create music is not that far from realising a book which is not meant to be read; the theoretical stance is the same. Bruno Munaris13 series of Libri illegibili (Unreadable Books) (fig.17) is a striking example of recovering objectivity. In his Da cosa nasce cosa, Munari says that he wants to explore the possibilities of visual communication within a book:

Normally when we think of books we think of texts of vari-ous genre [] to print on the page. A little interest is given to paper, binding, ink colour [], typefaces, and even less to blank spaces, margins, pages numbering, and all the rest. The aim of this experimentation has been trying to under-stand if it is possible to use the book material (text excepted) as visual language. Therefore, the problem is: [] Can the book as an object, regardless of printed words, communicate something? (Munari 2002, 217)

Munari is speculating on the recovery of an object and the discovery of it as a medium of communication. In his case, communication happens on the visual and tactile levels, whereas in Performairs case the levels involved are the visual and auditory.

There is something more to design than developing brand new things. Most of the time what for us is an original idea ends up be-ing something thought or invented by someone else already.

fig.17Libro illegibile (1948) - Bruno MunariDetail of a two-coloured spread obtained from a particular series of die-cuttings


To use old-fashioned devices for creating music, instead of throwing away something which still works, might seem nave. Nonetheless, the ethical and aesthetic sides of this matter originate in the ques-tion Munari was asking. Without this question there would have been neither Munari nor ready-made. The act of recovery and dis-covery, which I previously mentioned, can somehow be assimilated to ready-made: the capability of re-using objects, tools and devices in a new way so as to make the most of their communicative po-tential. Another aspect to take into consideration is why a book is commonly considered as a simple support for text. A convincing answer is given by Eric F. Clarke14 in his Ways of Listening, where he proposes an ecological approach to music perception, mainly based on James J. Gibsons15 theories. He writes:

The world is a highly structured environment subject to both the forces of nature (gravity, illumination, organic growth, the action of wind and water) and the profound impact of human beings and their cultures; and that in a reciprocal fashion perceivers are highly structured organisms that are adapted to that environment. (Clarke 2005, 17)

The definition of the world suggested above perfectly overlaps with the present situation, concerning Western society in particular. Marshall McLuhans16 The Medium Is the Massage a play on his famous phrase the medium is the message is enlightening on this matter:

All media work us over completely. They are so pervasive in their personal, political, economic, aesthetic, psychological, moral, ethical, and social consequences that they leave no part of us untouched, unaffected, unaltered.(McLuhan 1967, 26)

It appears clear, then, how humans, as perceptual systems, be-come overwhelmed by media and their continuous series of inputs. Attuned to this environment, we apply a filter to reality and are somehow forced to select. But this survival mechanism, almost un-conscious, implies the weakening of our sensorial perception. The perceptual spectrum of our senses seem to become less and less sharp. One example is the state of numbness in which the objectiv-ity of a book passes unnoticed: we no longer see the book but what is on the book. This observation can be applied to other familiar objects, such as an old printer: what would normally be perceived as background noise, could be structured and perceived as music. Performair is about selecting that particular kind of cognitive filter, which is an impediment to perception, and to re-establish a com-munication between the human and the artefact.

Before analysing the project, a short description of Gibsons ecological approach is essential to clarify how music is perceived. He explains that a system hunts until it achieves clarity (Gibson. 1966, 271). Nevertheless, clarity, today more than in the past, is a sort of mirage which is progressively slipping away:

Introduction 17

Organisms and their environments are constantly changing. [] Human beings have exploited natural opportunities for music making and have also adapted themselves to those op-portunities, and enhanced those opportunities, through tool-making of one sort or another []. Once made all these ar-tefacts help both to sustain existing musical behaviours and to make new behaviours possible. (Clarke 2005, 20-22)

By exploiting the hidden potential of office devices, Performair proposes a different approach to music composition and perform-ance.

This dissertation is articulated in the following sections.In chapter 2 the premises within the introduction will be fur-

ther developed. To locate the project within the fields of interaction design and installation art a series of case studies will be taken as reference, this to analyse the state of the art.

Chapter 3 will then briefly mention some key concepts about sound and auditory perception to make understandable the techni-cal side of the project treated in chapter 4. This chapter will be a guide through all the steps undertaken along with all the problems encountered in the realisation of the project.

Chapter 5 will then concentrate on the project itself, on what are the main instruments which compose Performairs unconven-tional orchestra and how they actually produce sound. The final chapter will summarize the whole thesis process to then imagine possible implementations and the future scenario of the project.

fig.18Casual Poetry (2009) - Authors ownA printed text from initial tests with an IBM dot-matrix printer. The result was not intended but had some visual quality to it

Research focusing and contextualisation 19

The discussion of ready-made in the previous chapter is strictly intertwined with both the misuse and the decontextualisation of objects.

The capability of merging the realms of interaction design and installation art by misusing decontextualised objects make the work of Rafael Lozano-Hemmer1 surely noteworthy. For this rea-son, I will now describe two pieces I saw during the 52nd Biennale di Venezia in 2007. Some Things Happen More Often Than All of the Time was the name of the exhibition curated by Pramo Lozada and Brbara Perea at the Mexican Pavilion which show-cased, among others, those installations.

Despite the different aesthetic tone, a common approach could be found in the development of affordance2. The physical charac-teristics of objects, which normally suggest their function(s), get reinvented and declined each time in a different way. To notice how Lozano-Hemmer also works on a very discreet and delicate mapping the relationship between user control and installation response. For the users to experience such installations, the artist must rethink how they could possibly interact with objects which have been distorted: a chair which does not act as a chair might have, on an adult, the same impact that a chair has on a baby the first time he sees it. As adults, we usually relate to the world by means of our experience and personal knowledge: a chair which does not act as a chair is not a chair anymore.

To accept something which goes against our beliefs, we, as us-ers or simply spectators, need a convincing metaphor which could please our mind and senses. Lozano-Hemmers methodology of

fig.1 previous pageRandom Plotter Art (2008) - Authors ownCombination of default patterns printed by a plotter during some ink tests. The person-al interest in random rhythms and patterns was prior to Performair

Research focusingand contextualisation


mapping and renovation of affordance appeared to me effective and totally convincing: objects for everyday use came to new life and surprised me for their natural behaviour.

2.1 A case study: Lozano-Hemmers method

Before describing all the further examples as a case study, I will tend to quote what the authors themselves write on their work to then develop my view. By sticking to the authors motivations, I believe that my speculation would be more trustworthy and in-cisive in giving Performair a contextual reference. The first piece to be presented is Wavefunction (figs.2,3,4,5,6), a reactive fluid-network of chairs premiered by Lozano-Hemmer at the Biennale:

Wavefunction is a kinetic sculpture comprised of fifty to one hundred Charles and Ray Eames3 moulded chairs (designed in 1948) and placed in a regular array of rows, facing the entrance to the exhibition space. When someone approaches the work, a computerised surveillance system detects their presence and the closest chairs automatically begin to lift off the ground, creating the crest of a wave that then spreads over the whole room. A system of electromechanical pistons raises each chair forty centimetres from the ground. The pis-tons are controlled by a computer that runs the mathemat-ics of fluid dynamics, thus making the waves interfere with each other, creating turbulence or becoming calm, just like real water. The idea of a function as a field for artistic ex-perimentation is a motivation for this piece. Other references

fig.2Wavefunction (2007) - Rafael Lozano-HemmerThe chairs in motion after the users pas-sage


fig.4 leftOverall view of the network of chairs in the static mode

fig.5 bottom leftDetail of the the surveillance system track-ing the users movement. In the top frame is possible to see how user is highlighted. In the bottom frame it is possible to see the ripples created by the users and the chairs which have been activated

fig.6 bottom rightRear view of the network of chairs in the static mode

fig.3 top leftWavefunction (2007) - Rafael Lozano-HemmerUsers triggering the surveillance system and initiating the movement of chairs


include: the mathematics of dynamic systems, capable of generating complex non-linear, behaviours, the materiali-sation of surveillance and turbulence and the anti-modular reinterpretation of the work of modern designers such as Charles and Ray Eames. (www.lozano-hemmer.com/english/projects/wavefunction.htm)

What Lozano-Hemmer describes as the motivation for the piece is what I was previously expanding on, the problem of function. Within Wavefunction, Eames chairs cease to be well known and recognised objects of industrial design to become something else. This stance somehow recalls that of Performair: printers, modems, scanners are not any more office devices but musical instruments.

Another aspect to analyse here is the re-interpretation of sur-veillance, which is a recurring theme in the field of interaction de-sign. Before getting into this, an introduction to what surveillance actually is appears necessary: surveillance is the monitoring of be-haviour, while systems surveillance is the monitoring of peoples behaviour in conformity to norms within a system, for reasons of security and social or political control.

Surveillance exemplifies a crucial matter, the concept of mis-use. According to this, the term mis-surveillance could describe a tracking system which, more than the simple monitoring of behav-iour, collects data for artistic purposes. What has to be underlined is that the misuse and the decontextualisation of an object are two sides of the same coin.

When used as a tool, every object automatically becomes a me-dium, for it establishes an implicit form of communication. Hence, a system of objects in this case a surveillance system can be intended as a system of media and, therefore, be subjected to some-

fig.7Pulse Room (2006) - Rafael Lozano-HemmerStaff member passing his heartbeat through the electric circuit to the first light bulb


thing definable as medium dichotomy. The medium is created for some specific purpose, though: it is the use or misuse of the medium itself that make its purpose. Both terms should be read positively: use means approaching the medium as it was meant to be used old function4 whereas misuse suggests a different way of ap-proaching the same medium new function5.

The next project by Lozano-Hemmer, Pulse Room (figs.7,8), is the first of a series called Pulse, including Pulse Spiral, Pulse Front, Pulse Park and Pulse Tank, which all present, in different scales, modes and arrangement, the transposition of users heartbeats into the pulsation of lights or even water:

Pulse Room is an interactive installation featuring one to three hundred clear incandescent light bulbs, 300 W each and hung from a cable at a height of three metres. The bulbs are uniformly distributed over the exhibition room, filling it completely. An interface placed on a side of the room has a sensor that detects the heart rate of participants.

When someone holds the interface, a computer detects his or her pulse and immediately sets off the closest bulb to flash at the exact rhythm of his or her heart. The moment the interface is released all the lights turn off briefly and the flashing sequence advances by one position down the queue, to the next bulb in the grid. Each time someone touches the interface a heart pattern is recorded and this is sent to the first bulb in the grid, pushing ahead all the existing record-ings. At any given time the installation shows the recordings from the most recent participants. (www.lozano-hemmer.com/english/projects/pulseroom.htm)

fig.8Pulse Room (2006) - Rafael Lozano-HemmerOverall view of the installation displaying with the series of lights all users heart-beats

Pulse Room is a plain example of what McLuhan intended when writing about how human beings are being progressively translated into media. The metaphor becomes physically evident, thus ceas-ing to be a metaphor: each heartbeat gets stored and then visually represented via light bulbs. No matter how technological the instal-lation is, Lozano-Hemmers ability is to transform technology into poetry. The warmth and blinking of an incandescent light commu-nicate well the essence of life; it possibly does better than the idea of a candle which inspired the author, as he wrote in his website. What happens, in reality, is just a sequence of ones and zeros i.e. the closing and opening of electric circuits but when staring at it, you are convinced those lights are persons. An effective metaphor can produce very vivid and intense emotions and make a clear dif-ference to how people approach an art piece.

2.2 State of the art

To picture the state of the art, let us consider first Dialtones: a tel-esymphony (figs.9,10,11,12,13,14), a project by Golan Levin6 and his large teamwork composed by Gregory Shakar, Scott Gibbons, Yasmin Sohrawardy, Joris Gruber, Erich Semlak, Gunther Schmidl, Joerg Lehner, and Jonathan Feinberg. The project was presented by Levin at Ars Electronica7 in 2001 and then at the Swiss National Exposition in 2002. The passing of time did not change the stance of this project and its related performance, which can still be con-sidered topical:

Dialtones is a large-scale concert performance whose sounds are wholly produced through the carefully choreographed dialing and ringing of the audiences own mobile phones. Because the exact location and tone of each participants mobile phone can be known in advance, Dialtones affords a diverse range of unprecedented sonic phenomena and musi-cally interesting structures. Moreover, by directing our at-tention to the unexplored musical potential of a ubiquitous modern appliance, Dialtones inverts our understandings of private sound [and] public space [...]. Wireless telephony has quickly become an indispensable aspect of modern life. [] Ironically, the astonishing eagerness with which we have adopted mobile phones is matched by our almost equal re-pulsion on the occasion of a cell phones ringing. [] An-nouncers at every modern-day concert command us to turn off our cell phones. [] The mobile phones speakers and ringers make it a performance instrument. The buttons make it a keyboard and remote control. Its programmable rings make it a portable synthesizer. (www.flong.com/storage/experience/telesymphony/index.html)

The need to revaluate objectivity already treated when writing about Munaris Libro illegibile associates Dialtones with Perfor-mair: the purpose is to organically make music with devices which,


in normal contexts, are considered disturbing for their undesired sound quality. The sound component of a device is not an impedi-ment to set aside but a quality to exploit.

figs.9,10 left, bottomDialtones (2001-2002) - Golan LevinDetailed view of the GUI and diagram of the installation



figs.11,12 right, center leftDialtones (2001-02) - Golan LevinMobiles loading in the roadcases - working as charging stations - and getting set for the performance

fig.13 center rightTeam member pictured while dialing the mobile numbers of the audience

fig.14 bottomOverall view of the live performance in a theatre. Members of the team are playing on stage and divided in three groups each il-luminated by a red, blue or green light. The audience can see its reflection and the di-aling patterns on the screens placed both sides of the stage


In The Universal Principles of Design, the authors treat a thematic which could be useful to go deeper in the auditory theme. Func-tional aspects of a design are less subjective than purely aesthetic aspects and, therefore, functional criteria present a clearer and more objective criteria for judgement of quality. (Lidell 2003, 90). Perhaps, this can explain how the sound quality of devices not designed for music making automatically becomes irrelevant. The sound samples surely are functional but secondary to the main function: the mobile was invented for facilitating a long distance communication combined with the possibility of movement.

Generally speaking, we tend to be reluctant to accept what is totally new versus what we are familiar with the new always presents old functions (McLuhan). However, all this has become less relevant in recent years: with the appearance of the iPhone an old device it is still a mobile phone presents new (innova-tive) functions multi-touch applications for music making and accelerometer based applications for orchestra direction (figs.15, 16). Particularly pertinent is also the problem of space which af-flicts office devices. Although printers, modems and scanners are less ubiquitous than mobiles they are used indoors it is difficult to conceive for them a different setting than a performance stage to explore their potential. This reflection leads us straight to other figures who used technological devices extrapolated from differ-ent working environments to perform music. The idea of making office devices perform music exists in a developing field of experi-mentation: people play with all sorts of objects, even floppy disks,

fig.15 bottom leftBravo Gustavo (2009) - Hello AgencyScreenshot of the iPhone application

fig.16 bottom rightExplanatory screenshot of the application which uses the accelerometer to translate the users movement into the direction of the orchestra. The faster the movements, the faster the tempo of performance


and hacking them in order to get sounds. To finally restrict the ground where Performair can be located, a triangulation between similar projects, all involving dot-matrix printers, will be made.

The first project to be analysed is Dot Matrix Patterns (figs.17, 18,19,20) by Metabiosis a duo composed by Aymeric Mansoux and Marloes de Valk who mainly investigate digital art, software, and the themes of music and image:

These graphics are not generated as an image and sent to the printer, but instead are [generated] directly using the 8-pin graphics mode of the printer. In this case, each of the eight bits in a byte of data sent to the printer corresponds to one pin on the print head. A bits value can be either 1 or 0. When the printer receives the data, it interprets a bit with a value of 1 as a command to fire the corresponding pin. Bits that are set to 0 dont cause pins to fire. Each block code will end up with its own byte id so it can be visualized and identified on paper in the graphics line. But before getting there, a few tests have been done to try different bit combos and select the generators the most aesthetically pleasing for us. (http://metabiosis.goto10.org/2008/03/25/dot-matrix-patterns/)

I find the visual result very convincing. Those abstract and geo-metrical shapes were obtained, as the artists themselves admit, by an intense experimentation. Hacking a printer to directly control the print head so as to have graphics is a matter of hard work. Afterwards comes a slow and systematic mapping between each command line and its graphical counterpart.

The procedure is far from Performairs, since here music is not involved at all. In this case the dot-matrix printer is simply used to print: it is a different way of printing aimed to different visual results, yet it is still printing. Metabiosis work can be framed in the field of PostScript Art: a series of visual experiments, made by different artists, based on PostScript language8. Concerning Per-formair, the reproduction of music generates the printing, so the difference is evident.

#!/bin/sh# OKI 320 ML tests# this test is to generate (crap) random 8 pin mode patterns#PRNGfunction rand {echo obase=8;èxpr $RANDOM % 256` | bc}function randpat {n=1while [ $n -le 256 ]; doRND=`randècho -ne \$RNDn=$((n+1))done}PATTERN=`randpat`# noise# PATTERN2=`head /dev/urandom -c 1024ècho -e ^[@^[5^[9^[*1\0\4$PATTERN^[@ | lp -o raw

fig.17Dot Matrix Patterns (2008) - MetabiosisExample of raw code used for experiment-ing a particular kind of pattern


figs.18,19,20Dot Matrix Patterns (2008) - MetabiosisSeries of visual patterns printed by 8-pin dot-matrix printers


The second project to consider is Paul Slocums9 Dot Matrix Synth 3.0 (fig.21). The work is an ongoing project since 2003, and consists in hacking a dot-matrix printer by reprogramming its firmware10. This operation turns a printer into a real music instrument and allows direct control of the device by means of a button interface. Slocum describes the sound generation:

The printer creates sound from the print head firing pins against the paper and the vibration of the stepper motor driving the print head back and forth. To generate differ-ent notes, the software adjusts the frequency of the printing process. Higher pitches tend to come from the firing of the pins against the paper, and lower pitches come from the rat-tle of driving the stepper motor. The external eight-button interface plugs into the printers font cartridge port. Each button has an assigned pitch, and pressing multiple buttons simultaneously activates the arpeggiator that quickly cycles through the notes you are holding down.

What makes this project more relevant than others is the relation-ship between the melodic and the visual sound of the perform-ance:

As the printer is played, its also printing a set of images that are programmed into the printers eprom with the software. There is interaction between the images and music. The im-age dithering patterns fluctuate depending on what notes are played, and the musics volume and rhythmic patterns change depending on the pattern in the current horizontal section of the image. (www.qotile.net/dotmatrix.html)

Nonetheless, there is a crucial point to be clarified. On the one hand, within Dot Matrix Synth, there is a reciprocal interaction between the making of melody and the printing of images it is a two-way process. On the other hand, within Performair, there is a direct interaction between what is computed and what is printed meanwhile it is executed it is a one-way process. Although the two projects from a superficial analysis might seem very similar, the theoretical stance behind them is completely different.

Moreover, what is being printed in Slocums case is an image whereas in my case it is a typographical score: the printed result of Performair has a visual rather than a figurative connotation. In a certain sense my approach can be described both as limited for it sticks to the device limitations since there is no hardware hack-ing but, at the same time, as coherent what is being keyed in is exactly what gets printed and heard. This point is essential. If it true that a device requires a much deeper effort than simply using it, it is also true that using a device implies a series of constraints and, thus, its peculiar difficulties. It could all be reduced to a matter of the aesthetical intent of making beautiful compositions versus the ethical intent of making faithful compositions sound as aestheti-cally valuable as possible.


fig.21Dot Matrix Synth 3.0 (2003) - Paul SlocumDetail of images printed during a live per-formance with an Epson dot-matrix printer


Surely, who proposes the closest idea to Performair is the Canadian duo [The User] architect Thomas McIntosh and composer Em-manuel Madan with their Symphony #2 for Dot Matrix Printers (figs.23,24,25), a work focused on ambient noise:

The Symphony focuses the listeners attention on a nearly forgotten technology: the dot-matrix printer. Specifically, it employs the noises the printers make as the sole sound source for a musical composition. Leaving the constituent elements untouched, the process imposes a new order upon them, re-organizing the sounds along a musical structure. Dot matrix printers are thus turned into musical instruments, while a computer network system, typical of a contemporary office, is employed as the orchestra used to play them. The orches-tra is conducted by a network server which reads from a composed score. Each of the printers plays from a different part comprised of rhythms and pitches made up of letters of the alphabet, punctuation marks and other characters. [The User] uses ascii text files to compose, orchestrate, and synchronize sonorous and densely textured, rhythmically-driven music. During the half hour performance, the sounds are amplified and broadcast over a sound system. (www.theuser.org/dotmatrix/projinfo/en/frame_index.html)

This project is similar to Performair, but again there are some deep differences. The experimentation of the duo is oriented only to-wards dot-matrix printers whereas my research includes also other office devices, such as modem and scanners.

Secondly, to have each printer managed by a computer and then control all computers with a server it is, both on the hardware and the software level, a completely different stance from Perfor-mairs. The technological challenge within Performair it has been experimenting with several typologies of device and trying to real-ise a system to manage them all through one computer only. It is a different way to conceive the same problem.

fig.22Symphony #2 (1998) - [The User]Series of dot-matrix printers set to perfrom


figs.23,24Symphony #2 (1998) - [The User]Team double-checking the system before the performance (top) and detail of the amplifi-cation of a printer (below)


2.3 Sampling: an alternative scenario

Now that the state of the art has been proposed and the location of Performair been clarified, it is possible to move forward towards the realm of music. Both works are based on the same simple as-Both works are based on the same simple as-sumption, the possibility of composing music by recording audio fragments of technological machinery while they are in use. With-out divagating too much in describing this experimental field of music there are far too many artists, tendencies, genres and labels to examine I would take the two full lengths as an excuse to figu-rate an alternative scenario to Performair.

When Machines Exceed Human Intelligence (fig.25) is the de-but full-length which Harmonic 313 released under the label Warp in 2009. Behind the moniker is hiding Mark Pritchard, one of the most acknowledged and active musicians in electronic music with a numerous series of projects Global Communication, Jedi Knights, Harmonic 33 and alias Troubleman, Link, Reload among many others. The album is a mashup of all kinds of obsolete devices and sounds coming straight from arcade games, a sort of travel through the long history of consoles but not only. Aesthetically speaking, this composition of machine-made audio samples sounds astonish-ing and was an inspiration to Performair. The ethic of my work is obviously different, however: a project about music, like Perfor-mair, cannot leave aside the aesthetic value of music composition. In this sense Pritchards pieces, Quadrant 3 in particular, are an inescapable reference.

The other work I was previously referring to is A Chance to Cut Is a Chance to Cure (fig.26), an album released by the experimental duo Matmos M.C. Schmidt and Drew Daniel in 2001 under

fig.25When Machines Exceed Human Intelligence(Warp 2009) - Harmonic 313Album front cover

Matador Recordings. In the genre of electronica few can compare to Matmos who are able to structure an entire full-length around the medical world of machines and cosmetic surgery. If there is a boundary in music here it has been shifted much further: not only equipments for liposuction, rhinoplasty and anaesthesia are sam-pled but also the sound of teeth, bones and reactions to the surger-ies. Hard to believe but the result is pleasant, even attractive, and demonstrates how from a radical and unlikely stance remarkable music can be composed.


fig.26A Chance to Cut Is a Chance to Cure(Matador Recordings 2001) - MatmosAlbum front cover

Paul Hindemith said that music, in any form, is nothing but noise without meaning till it reaches a mind capable of receiving it. (Ott Krolyi 1980, 201) Except for particular circumstances, the discus-sion about music in this chapter will relate to concepts rather than physics or anatomy.

3.1 Sound

It is impossible to discuss music or a musical project before intro-ducing its vital premise, sound. Sound is nothing but a vibration of a medium air, water, wood etc. by means of which this vibra-tion can be transmitted. A regular vibration results in a musical sound, characterised by a given pitch, while an irregular vibration produces simply noise.

The only motion which can originate sound is vibration: what-ever the instrument, some components always vibrate strings, drumheads etc. and others amplify this resonation. The vibration originates waves which move through the air by a series of com-pressions (high pressure) and rarefaction (low pressure) of the air molecules: to picture the phenomenon you can think of sea waves. The speed of sound propagation in air is approximately 335 m/s depending on atmosphere variations. Waves are defined by speed, length and frequency (fig.2): the speed v indicates how fast a wave moves from a point a to point b, the length l is the distance be-tween two sequential wave peaks, the frequency f is the number of oscillations in a second and is measured in Hertz (Hz).

Sound and auditory perception 37

Soundand auditory perception

fig.1 previous pageSheet paper - Authors ownConventional music notation

The modulation of a waveform is of two kinds: amplitude modu-lation (am) is the size variation of wave peaks, while frequency modulation (fm) is the size variation of the wavelength (fig.3). The terms am and fm are commonly used to describe how radio signals travel. In music, a regular amplitude modulation is commonly de-fined as vibrato, a regular frequency modulation tremolo.

The three characteristics of a sound are pitch, intensity, and qual-ity. Pitch describes our perceptive capability to distinguish between high and low sounds and is related to the frequency: the higher the frequency the higher the sound and vice versa (fig.4). Human hearing works between a minimum of 16-20 vibrations per second described with the notation c/s where c stands for full cycle and a maximum value of 20,000-25,000 c/s. Musical instruments work within this range. Intensity measured in dB (decibel is dimension-less) or W/m2 (Watts per square metre) is related to the amplitude of the vibration: the higher the amplitude the louder (more intense) the sound and vice versa (fig.5).


l

Atime

fig.4Pitch - Authors ownComparison between a higher sound (above) and lower sound (below)

fig.3Modulation - Authors ownComparison between amplitude modulation (above) and frequency modulation (below)

fig.5Intensity - Authors ownComparison between a louder sound (above) and a weaker sound (below)

fig.2Wave measures - Authors ownIllustration of the length and frequency of a soundwave


Quality defines the colour of a note and allows us to perceive the difference produced by the same note played by different instru-ments. Since a note corresponds to a given frequency all instru-ments should play the same, but frequency is only the primary characteristic of notes. The secondary characteristic, no less im-portant, is harmonics which determine not only the timbre but also the vividness of notes: the variation of harmonics intensity make the difference. Consonance an interval or chord which produces a sense of stability and dissonance an interval or chord which produces a sense of instability are strictly intertwined with har-monics. According to the Helmholtz1 theory of consonance, conso-nance or dissonance depend on how many harmonics have the note within the same interval: the more harmonics in common the more consonance; the fewer the more dissonance (fig.6).

13

12

11

10

9

8

7

6

5

4

3

2

1

13

12

11

10

9

8

7

6

5

4

3

2

1

primary note

harmonicseries

oboe horn

fig.6Harmonics - Authors ownComparison between the harmonics of an oboe and a horn


3.2 Auditory perception

The psychological study of the human perception of sound is called psychoacoustics. Its aim is to explain how sounds get identified and extrapolated from the environment, and decoded by our ears to be, only then, elaborated by our brain.

Information within sounds is transmitted by means of a spatial and temporal sequence: obviously this applies to the outer environ-ment but also to music. A melody can be identified through the intervals which link different notes and make them perceivable as a meaningful whole. Tension and pause, together with the chosen notes, are the ingredients of a melody. Without the rhythm deriv-ing from tensions and pauses there would be no melody, just a meaningless phenomenon without shape.

As Ott Krolyi writes in his Introducing Music, to be effective a melody needs balance, which is commonly reached by means of recurring themes. A distinction is necessary to clarify the signifi-cance of theme as a term. A scale a series of ordered notes which equally starts with a lower octave and finishes with a higher octave or vice versa is not a theme. It is simply the frame which allows the improvisation of a melody in a tonality a system of notes subjected to a hierarchic tone or the modulation between differ-ent tonalities. Any interval, melody or chord can be translated in a different tonality. A basic example of a musical theme is the use of a swelling movement to which corresponds a subsequent decreas-ing movement.

The practise of harmony the simultaneous combination of two or more sounds appeared only later in the beginning of the 9th century. While melody has a horizontal development, the harmony has a vertical development. This will be covered later in Section 5.6 when comparing the traditional staff with Performair score.

The auditory system is in charge of interpreting all the intrinsic connotations of sounds frequency, duration, speed and the rela-tionship among them. Melody is a sort of benevolent virus which is constantly changing: what we take for granted as it happens au-tomatically hearing is the result of the constant adaptation to a constant change. Clarke reinforces this idea:

In everyday life, the perception-action cycle is usually so seamless that there may be little need or opportunity for per-ceivers to become aware of their subject-position in relation to the world. [...] The overwhelming majority of perceptual learning occurs passively, [there is] no explicit training in-volved, [but it is] profoundly active from the perspective of the organism itself. (Clarke 2008, 23, 124)

It should appear much clearer now the question of mutual adapta-tion treated in the Introduction.

Given two sources able to vibrate with the same oscillation, the one subjected to direct stimulation would sympathetically in-fluence the indirect vibration of the other. As the oscillation of a string is reinforced by the resonating chamber of the instrument,

the oscillation of a vocal cord makes all the upper part of our body resonate. As perceptive beings we also resonate to the outer en-vironment (fig.7). The auditory canal has its peculiar resonance [] which makes us more sensible to frequencies between 500 and 4000 Hz. (Grassi, Plack. 2008, 12).

Sound and light waves are similar in some respects. As light can directly hit an object but also be reflected from that object towards other objects, so can sound. It is somehow less credible simply be-cause we are counting too much on the visual side of perception to the detriment of the auditory.

Relevant to the discourse is intonation, the pitch accuracy of an instrument. In an orchestral performance, non-tuned instruments playing together instead of a symphony would produce a cacoph-ony. Intonation prevents this by setting two or more instruments to the same note i.e. a given frequency (conventionally 440 c/s corresponding to A) so as to produce a unison the replication of the same note.

A couple of brief annotations. The majority of office devices cannot be tuned for they were not meant to play music. Not only the defect of intonation affects office performances, but also the acoustic of the environment itself: non-reciprocal sounds are repeat-edly bouncing within our four walls. The design studio, considered as our auditorium, has its own resonation period corresponding to the time needed for a sound to fade out. Most auditoria are char-acterised by a resonation period of approximately 1 to 2 seconds which is surely not comparable to our case for evident reasons.


fig.7Ear resonance - Authors ownIllustration of the auditory canal resonating

Concept definition and methodology 43

For my thesis, I was keen on a music-related project within the field of interaction design. Personally speaking, music is not only a matter of pleasure and cultural enrichment but also one of the highest art forms. If I were to explain in a sentence what interaction design has to do with music, I would say that it is about enlarging the boundaries of music and amplifying its communicative power. New instruments can be designed along with new ways of interac-tion within them. Most relevant to my thesis, new ways of music making with non-musical instruments can be conceived and pur-sued, but this I realised later when focusing the research.

The first idea concerned a portable object which could enhance the listening experience of a user: a pair of headphones which could allow real-time mixing and a three-dimensional perception of mu-sic. In order to design such a device I sought similar devices avail-able on the market. The more I focused on the subject the more I discarded elements which at the beginning appeared relevant but, in the final instance, were not.

I did not want to create something new. I wanted to do some-thing new with something old: a reason might be my attention to recycling and sustainable design. Thus, I started thinking how everything that surrounds us has a musical tone, no matters how noisy or unorganised this might seem. Perhaps there could be a structure some basic rules which could make sounds and noises identifiable, understandable, and, therefore, aesthetically valuable for our ears and minds. I also noticed how this auditory process of comprehension is much easier in a natural than in an artificial environment: to find music by the seaside it is surely less hard than

Concept definitionand methodology

fig.1 previous pageDot-matrix scores - Authors ownSuperimposition of all the test scores print-ed with different dot-matrix printers


in a city centre; natural dynamics still have their hold on us. As a designer I then speculated how this reflection could be translated into a much smaller scale, and this led me to think of a design studio or, more generally, a simple office. I therefore imagined a space where devices were playing as if they were reading an invis-ible score, a studio becoming a theatre and the design process a performance.

4.1 Project scope: Verplanks method

In order to frame Performair, Bill Verplanks1 four-step process (fig.2), is very useful in defining the interactions involved within the project.

1. MotivationWhat, in a design studio, is normally perceived as noise could be meaningful. The environment where designers usually work could be alive and, therefore, lived in a different way. There is an osmotic relationship between humans and environments as well as a mutual communication what I would define as a ping-pong effect be-tween users and objects.

What the user is seeking, as McLuhan theorized, is the experi-ence of being translated into the media; this is the real interac-tion. Media are seen as extensions of our body; this is attractive because it triggers sub- and unconscious reactions, both on mental and physical.

2. MeaningFrom the need to make music in a tangible way, office devices be-come musical instruments metaphor while the studio becomes the theatre scenario where this orchestra can perform musical compositions.

3. ModeThe Performair set-up resembles a small chamber orchestra whose task is to perform an innovative piece of music. To do so, all the peripheries are connected via powered hub to a computer. By click-ing on the keyboard, the different devices react to the input and perform their part of the composition.

4. MappingRegarding the displays and controls, the system is an hybrid be-tween a Graphical User Interface (gui) and a Tangible User Inter-face (tui):

- the gui is a piece of software developed in QBasic2, i.e. the infra-structure that controls all the peripheries- the tui is the computer keyboard, used to control the above-men-tioned system by typing letters which initiate command strings

fig.2 following pageMotivation and metaphor - Authors ownIllustration based on Verplanks method


4.2 Preliminary phase

Massimo Banzi3 writes in Getting Started with Arduino:

One of the best ways to quickly get to results is to find a great source of technology junk and use it to quickly (and cheaply) get to prototype an experience. Accumulate junk and go through it before starting to build something from scratch. (Banzi 2008, 12)

My initial approach to the pre-prototyping phase somehow resem-bled Banzis words. However, even getting to the junk I needed ap-peared to be a problem for I was seeking office devices, dot-matrix printers in particular.

According to the latest Italian laws on the ecological matter, technological devices are classified by competent institutions as raee rifiuti di apparecchiature elettriche ed elettroniche (fig.3). Special material, even if rubbish, cannot be sold or lent to a citi-zen but only listed and disposed for recycling in appropriate scrap-yards. This meant that public institutions such as councils, hos-pitals, schools, devices left on the floors, would not allow me to have them legally. Because of this, I first started to search in specific scrap-yards around Friuli Venezia Giulia and Veneto, then in com-puter centres, and finally in companies and courier offices where dot-matrix printers are still in use (see Appendix B). By building a network of contacts I managed to gather far more printers than I needed eventually. Afterwards, all the devices had to be cleaned and tested (fig.3) in order to see which was working and which had to be fixed or brought back to the ecological scrap-yard (fig.4)

fig.4 bottom rightDot-matrix circuits - Authors ownGetting dirty with dot-matrix printers to fix and test them

fig.3 bottom leftraeeEcological scrap-yard where special materi-als get collected and then recycled

To understand the situation from different perspectives I started researching on my own but also sought advice from professionals in the field of informatics, physics and engineering. I first visited Enaip4 and then the Centre of Informatics and Engineering at the University of Udine (see Appendix A). There I could show to com-petent professionals my thesis and its main technological challenges to get relevant feedback. The consultations, along with a series of initial tests, were helpful to focus on the main problems involved in the actual making of Performair.

4.3 Hardware challenge

There are two kind of ports to connect a dot-matrix printer to a computer:

- Parallel Port (lpt) (fig.6)- scsi Port (scsi) (fig.7)

After researching, the best choice appeared to be the first one, i.e. the parallel port. However, this implied a complication for no board is available on the public market which would be able to expand the parallel ports of a computer to the actual number of devices I was aiming to use.

Ideally, the devices had to be divided into three groups - made of two units each - according to my studies of orchestral compo-sition, but also of the practicality of the final performance itself. Another choice then had to be made so as to control more than one device with the same computer:


fig.5raee recyclingMachinery used to recycle special materials


- Building a handmade parallel port multiplier- Using a powered usb hub (fig.8)

The second option appeared to be more convenient for it was faster and easier to achieve.

The only element which concerned me was the possibility that a parallel-to-parallel connection had a pin-to-pin correspondence while the usb-to-parallel connection obviously not. I thought this might have been a concern, but it eventually was not because ap-peared to be the sole option I had anyway. The procedure would then be to connect one printer at a time via usb-to-parallel adaptors (fig.9) and, afterwards, a series of 56 K modems. Crucial would be to have the least latency possible and the most accurate synchrony between all the instruments. This depends on the software side of the project.

fig.6lpt male - Authors own

fig.7scsi male - Authors own

fig.8Powered hub male - Authors ownIt is used to connect all the peripherals to one computer

fig.9usb-to-parallel adaptor - Authors ownIt allows the operating system to manage more than two peripherals, since the default number of lpt ports on Windows bios is ei-ther 2 or 3


4.4 Software challenge

How to control a series of devices in perfect synchrony? For the accuracy and effectiveness of the performance even milliseconds of latency should be avoided, and this is obviously difficult. Consider just printers as an instance, even though this problem applies to any peripheral. Simply sending more than one text file to different printers is not a solution because no operating system would give a real-time response. Windows has its buffering time to execute a user request, and that is not accurate nor computable: there would be no simultaneity if we let the operation system manage the print queue. After checking several forums on the matter, the solution was analysed with some engineers. I first proposed what Paul Slo-cum did with his Dot Matrix Synth 3.0 (fig.10) where he made an external controller as an interface to send data to the printer via firmware.

To send command strings by firmware, as Fabrizio Barbarino5 explained to me, implies a deeper understanding in programming terms: designing a piece of software to dialogue with the paral-lel port of a device does not mean an assured result, still less the desired one. Between the software and the device is an operating system in this case Windows and its drivers, which complicate the data transmission. The platform has to be by-passed to achieve that given action which produces that given sound.

fig.10Dot Matrix Synth 3.0 (2003) - Paul SlocumHacked Epson dot-matrix printer controlled by a custom interface able to send string commands via firmware to the device


It is necessary to set the parallel ports from in such a way to send data in raw spp6 or epp7 mode (figs.11,12,13,14). In other words, avoiding other low levels of control to send just simple electric im-puts so as to get precise results. This is why Slocum modified the firmware of his printer, not only to make the printer sound as he wanted but also to avoid problems such as the printer jamming for the lack of paper.

Different programming languages from the most basic to the most advanced could be used to realise Performair: dos, Pascal,

c, Basic.

fig.12Raw mode setup - Authors ownTo bypass Windows control, it is necessary, for testing, to make sure the data can be sent to the printer in a safe way

fig.11lpt port setup - Authors ownSince the computer used for early tests has only one parallel port all the printers have been installed on lpt1


figs.13,14Raw mode setup - Authors ownTo bypass Windows control, it is necessary, for testing, to make sure the data can be sent to the printer in a safe way


4.5 From pre-prototyping to prototyping

The pre-prototyping stage consisted of a series of systematic and non-interactive tests based on a step-by-step procedure. Firstly, every single device had to be played to verify its musical timbre or range and versatility, i.e. the possibilities of producing particular noises.

Similarly to electric, semi-acoustic or acoustic instruments, a printer, a modem or other technological device sounds slightly or totally different one from another. The difference depends on sev-eral factors: materials, assemblage, technology, and state of usage. An old instrument never sounds like a brand new one, its sound will always be peculiar. In this sense, there is not much difference between a musical instrument, say a guitar, and a technological device such as a dot-matrix printer.

Dot-matrix printers are versatile devices because typographic elements are printed on paper by a matrix of pins - 8, 9, 18, 24 (most common) or 36 - which differs by model and manufacturer. This implies a strict correspondence between the visual input and auditory output, very relevant to this project. It is like composing for a classical instrument: a violinist plays notes on a treble clef by reading a score while a dot-matrix printer plays notes by writing a typographic score. Different logical process but equal result: music.Secondly, for every device a table of correspondence between typo-graphic elements and sounds had to be mapped (fig.14). The more similar the typographic element the more similar the sound: a v and a w are visually similar, meaning that the needles to be activated would produce a similar sound. For the same reason, a v and an i would sound different.

Finally, a series of short pieces was composed as simple text files to get started with performing. This step was essential to under-stand what problems could happen when playing live and impro-vising with more than a device (figs.15,16). Thanks to the help of Barbarino, a second series of interactive prototyping tests was run on a raw QBasic software; a simple program which would print a simple message. Afterwards, a more refined version was developed to use the keyboard as medium of interaction with the printer: by pressing a key the printer would print a specified symbol resulting in a particular note (figs.17,18,19,20).

These tests aimed to manage simultaneously a series of devices to design a more reliable program: initially a single device has to play, then other devices have to play in perfect synchronous. Mean-time, some more research was done on the type of music to com-pose and possible ways of performing it. Clear at this stage was the necessity of a fully working program able to handle different types of devices from different manufacturers.

Lack of time could be a possible drawback in realising such an ambitious project. The worst-case scenario would imply system-atic sampling8 of notes - corresponding to printed letters, numbers, symbols (printers) and dial numbers (modems) - with Steinberg Cubase9 to perform live using professional music software such as Ableton Live10.

fig.15 following pageTonality test #1 - Authors ownText written to test the timbre of each print-er and to map correspondances between frequencies and typographic elements


...............................................................................................................

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

---------------------------------------------------------------------------------------------------------------

===============================================================================================================_______________________________________________________________________________________________________________

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

***************************************************************************************************************

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

///////////////////////////////////////////////////////////////////////////////////////////////////////////////

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((

)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{

}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}

>

???????????????????????????????????????????????????????????????????????????????????????????????????????????????

$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&

###############################################################################################################

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222

333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333

444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444

555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555

666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666

777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777

888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888

999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb

BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB


Movement I

__________________________________________________________________________________________________________________________________________________________________________________________________________________

!!!!!!!!!!!!!!!!!!!!__________________________

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! *!!!!!!!!!!!!!!!!!!!!__________________________

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! *$$$$$$$$$$$$$$$$$$$$%%%%%%%%%%%%%%%%%%%%__________________________

%%%%%%%%%%%%%%%%%%%%$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ *$$$$$$$$$$$$$$$$$$$$%%%%%%%%%%%%%%%%%%%%__________________________

%%%%%%%%%%%%%%%%%%%%$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ *

tempo (BPM)

Strophe A

Strophe B

Strophe B

Strophe A

Strophe D

Strophe C

Strophe D

Strophe C

A continuous line gets printed in one gowhile other elements in two

The star is used to fire the print head two timesto keep the song tempo

Each typographic element produces a frequency corresponding to a note; a repeated element produces a continuous note

The time to print the continuous line (1 go) and the star (2 gos)has been calculated to keep the tempo costant

Untitled #5 / minimal technoone voice piece for dox-matrix printers tested on Epson FX-850 and IBM 2391-001


figs.18,19,20,21 below, following pageQBasic Prototype - Authors ownBy pressing a tab on the keyboard the pro-gram sends to print the given typographic element so as to obtain the desired note

fig.16,17 previous pagesPattern test #4/Composition #5 - Authors ownText written to detect useful rhythmic pat-terns for dot-matrix compositions. A mini-mal techno piece for a dot-matrix printer

5.1 Why Performair?

Within Western music, performance is mostly considered as an in-terpretive art which usually include one or more performers those who play the music and an audience those who listen to the music. Although in the present time we tend to consider it as an interpretative phenomenon, music and musical performance origi-nally coincided: the pleasure deriving from music was the same pleasure deriving from the making of it since music started as an oral tradition of religious rituals.

A performer is not just a player, he is also an interpreter and throughout time this fact has not changed. In the past, the heart-beat and breathing together with the musical possibilities of human voice were the elements of music. Presently, the performer to a cer-tain extent influences the music he plays by determining its dynam-ics and phrasing through instruments. The more musical notation developed the more performers acquired the role of interpreters by being subjected to a series of limitations: the information within the score, their knowledge and technical skills but, more important, their taste. In this sense, the passage of time amplified the value of performance: it is not just a matter of enduring musical traditions but also a refinement of those traditions.

There is no performance without a performer and a medium to make it happen. In fact, to try to perform without air would be like trying to swim without water. Air is the medium which makes the magic: the act of performance suddenly becomes real visible and audible while the performers role becomes culturally valuable.

Performair: final project 59

Performair:final project

fig.1 previous pageArditti Quartet - Authors ownMember of the Quartet performing live at Exit_01 Party Sintomatico (Venice Biennale Musica 2008)


5.2 Orchestras

The previous chapters explained how office devices can be con-ceived as musical instruments. It is now possible to extend the met-aphor to the orchestra (fig.2). In the ancient Greek theatre this term designated an area between the stage and tiers where the chorus could dance and, more relevant, the musicians could perform.

As an instrumental ensemble, a modern orchestra is composed of five different sections woodwinds, brass, percussion, keyboards and strings and has a conductor. Some sections are in charge of the rhythm while others which develop the melody of the composi-tion. For this reason the Performair orchestra (figs.3,4) is divided into three sections:

- Rhythmic: dot-matrix printers and scanners- Melodic: dot-matrix printers and modems- Visual: dot-matrix printers and scanners

Within this ensemble the boundaries between different sections are not rigid but can be traversed. Dot-matrix printers, in particular, are very versatile and can play both rhythmic and melodic parts. Printers are also involved in visuals since they produce printed results while performing. Concerning modems, the peculiarity of producing different sounds according to the different numbers be-ing dialled makes them perfect for melodic parts. Scanners, on the other hand, are more suitable for rhythmic parts and can also be used for visuals. So what are their musical qualities?

fig.2 bottomOrchestra - Ott Krolyi

figs.3,4 following pagesPerformairs orchestra - Authors ownDiagram, overall view and details of the or-chestras sections

pcbrain

hubheart

modemmelody

dot-matrix printersrhythm + melody + visuals

scannersrhythm + visuals


5.3 Instruments

a. Dot-matrix printersAccording to the model and manufacturer, a dot-matrix printer can have print heads of from 8 to 36 pins, even though the most com-mon are 9-pin and 24-pin. As the matrix of pins varies so do the printed dots: 8 pins 8 dots, 24 pins 24 dots. Pins are arranged in vertical rows the 8-pin and 9-pin models have only one and are controlled by magnets. At each movement of the print head across the page a vertical selection of pins impacts first the ribbon and then the paper (figs.5,6).

There are as many bits of information as there are pins: 8 pins 8 bits, 24 pins 24 bits etc. Since a bit value can either be 1 (mode on) or 0 (mode off), the corresponding pin can only be activated (mode on) or deactivated (mode off). In the case of a 8-pin printer the data sent from the computer is encoded as a byte (8 bits) whereas for a 24-pin model 3 bytes (24 bits) are necessary to encode the data. To make a simple example, if we were to use a 8-pin printer and the information sent to it corresponded to the numeric value of 10,000,000 this would result in only the first pin being fired. What is relevant here is that the larger the number of pins the more versatile is the printer in printing and, therefore, as a musical in-strument: a 36-pin printer has a wider range of sounds than an 8-pin model for the pin combinations are more. This means a huge acoustic variance between dot-matrix printers and inkjet printers: older device are musically better than newer ones.

fig.5 leftPrint head - Authors ownIllustration of a 8-pin print head firing the pins to compose a P

fig.6 below8-pin print - Authors ownPrint mode of the letter A within an 8-pin dot-matrix printer


b. ModemsDialup modems are much like telephones and require a series of procedures before establishing a connection. Normally, a modem must be connected to telephone line and needs another modem at the other end of that line to reply. When connecting, the two mo-dems home modem and Internet Server Provider (ISP) modem initiates communication through the so-called handshake. Before exchanging information, the speed of transfer must be agreed, for each modem has its own standard: the model I used for tests is a V.90 USRobotics (fig.7), where the V.90 wording indicates a trans-fer speed of 56 Kbps (56,000 bps). I chose this rather than older standards mainly for the sounds it produces I was attuned to them but also for its availability.

How do modems produce sound? While computers are mostly based on digital technology, phone lines partly work on analogue technology. Modems function as signal translators by turning digital into analogue data which travels as electricity through the wires and produces sounds. It is called a modem because first it modulates the signal from digital to analogue then it demodu-lates it from analogue back to digital and the other way round to have a two-way communication (fig.9).

All modems respond to a command language called at a command line prefix meaning attention which is generally used to change parameters and setting within the device. Although every manufacturer and model differs, some command strings are uni-versal and can be used to produce sounds. Without the need of programming it is possible to access the modem via HyperTerminal a sort of white dos window where those at commands can be typed in. Here is a list of commands:

fig.7USRobotics 56 K modem - Authors ownModel used for tests

This system would not be very reliable in a live performance set-ting. Sometimes the HyperTerminal window does not show the command lines even though it executes them, and at other times command lines do not get executed at all. After all we are interfac-ing with the modem through an operating system and that always brings complications.


ATI3: Device identification and confir-

mation of AT commandsATA : Answer an incoming callATDT: Dial the following phone numberATE : Turn echo offATH1: Hang upATH0: Receiver modeATm0: Switch loudspeaker offATO : Return to on-line stateATZ : Reset the modem to the values stored in the NVRAM (Non-Vola- tile Random Access Memory)

+++ : Return to the command state (not to be preceded by )

/A : Repeat last command (not to be pre ceded by or followed by

fig.8Data modulation - Authors ownIllustration which displays how a 56k mo-dem works


c. ScannersSome peripherals, only for the sake of designers, feature hidden configurations which in jargon are defined Easter eggs. Some models of Hewlett-Packard (Scanjet 3c/4c) are provi

Documents

Performair: music for offices