0 1

During the last decades the scientific fieldof artificial intelligence has undergone amajor change in direction. The generalcognitive paradigm of constructing anartificial humanoid thinking machine wasbecoming more and more problematic. Itturned out that creating an intelligent robotcapable of communication is not just amatter of computational power. As a directanswer to this the 90s saw the rise of apromising and remarkable research domainin artificial intelligence called affectivecomputing. In this field, originally foundedby Prof. Dr. R. Picard at MIT, researchers arestudying innovative emotional man-machineinteractions. From these efforts a newbreed of humanoid robots including theKISMET (first generation) and Nexi (secondgeneration) was developed at MIT.

The groundbreaking vision affectivecomputing offers has also had a profoundinfluence on the field of computational andcognitive musicology. Specifically it hasrenewed interest in the connectionsbetween music and emotions. Since thetentative studies of K. Hevner in the 1930sthis connection could only be thought of interms of classical psychology. But byapplying the field of affective computing awhole new perspective opens up.

This point of view lies at the foundation ofthe construction of the EMO-Synth. Inimplementing the system numerous newprinciples and ideas have also come to

Artist Statement

02

surface. Two seminal ideas in this contextare the following:

+ The EMO-Synth can be seen as a brand-newvirtual tool by which the artist can extend andcomplement his own creative process. By usingthe system it becomes possible for the user tounderstandhis own personal creativityon a newlevel.

+ By using the system a totally new virtualplatform arises inwhich the boundaries betweenartist, audience and the generated artifacts areredefined. To what level are these artifacts acreation ofthe artist or the audience? To whichextent can we speak of artistic input when theEMO-Synth truly participates in the creativeprocess?

Of course the last two statements start froma certain perspective on the artistic process.It is my personal point of view that artcommunicates through human emotions.Every work of art implies a certain kind ofexpression that can be understoodintuitively rather than rationally. It is thisintuitive and emotionally loaded componentthat the EMO-Synth tries to uncover and

simulate. The realization of the EMO-Synthproject also implied a radical break with theold notion of static art. Doing this is not anew idea but it was only through recentdevelopments on both the methodologicaland technical level that the implementationof systems like the EMO-Synth becomespossible. Recent artificial intelligencetechniques and techniques stemming fromaffective computing give a whole newinsight into both the esthetic or emotionalcontent of art objects.

In developing the EMO-Synth I feel it as mypersonal goal that the project will redefineand reshape part of the whole newdiscussion about the place of automaticallygenerated art in the general artisticdiscourse.

Valery VermeulenProducer & Founder of the EMO-Synth

03

The EMO-Synth is an interactive multi-media system where the emotional man-machine interaction plays a central role.During performances and demonstrationsthe system automatically generates andmanipulates sounds and images to directthe user in certain predefined emotionalstates. The development of the EMO-Synthrelies on different techniques stemmingfrom a broad range of domains includingartificial intelligence, affective computing,psychophysiology, automatic musiccomposition and sound and imagegeneration.

In the current setup emotions arequantitatively measured using biosensorsthat monitor psychophysiologicalparameters such as heart rate (ECG orelectrocardiogram), stress (GSR orgalvanic skin response) andelectrooculogram (EOG). During operationthe EMO-Synth will subsequently generateaudiovisual artifacts and analyze theresulting emotional impact on the user.Using techniques stemming from artificialintelligence like interactive geneticprogramming and HMM’s the EMO-Synthwill try to generate audiovisuals to bringthe user in a specific emotional state.

The concept of the EMO-Synth is based onthe link between the apparent rationallogical and creative process. Recenttechnological and scientific experimentshave provided mounting evidence of thislink. As a consequence more and moreopportunities arise for the artist to usescientific systems in his own creativeprocess. I first came into contact withthese ideas during a four-year researchproject I undertook between 2001 and2005 at the Institute for Psychoacousticsand Electronic Music (IPEM, University ofGhent). These were a primary inspirationto start the EMO-Synth project.

EMO-Synth

04

05

Prototypes

Between 2004 and 2007 the first fourprototypes of the EMO-Synth weredeveloped to explore the possibilities of thesystem and to design possible blueprintsfor future developments.

Prototype 01 was developed for the multimedia project SKANNER (collaboration withBelgian artist Angelo Vermeulen) thatmainly dealt with the concept of human fear.The project was shown on various locationsincluding the Atomium (expo Interface,Brussels, Belgium, 2004), W139 (Amsterdam,Netherlands, on invitation of Belgian artistJan Fabre, 2004), Beursschouwburg(Subliminal Festival, Brussels, Belgium,2005), STUK (Artefact festival and Kulturama2005, Leuven, Belgium, 2005), Z33 (expo Feelthe young, Hasselt, Belgium, 2004). With thisprototype first experiments wereelaborated in the field of the sonification ofthe psychophysiological data.

Prototype 02 was developed in 2006 for theexposition PERSEVERANCE at the Godshuisin Sint Laureins in Belgium. A first automaticmusic generating system was implementedwith this prototype. During the expositionan EMO-Synth performance wasbroadcasted live on the national Belgianradio station Studio Brussels.

Prototype 03 developed new possibilities toextend the generating capabilities of theEMO-Synth. The music generation enginewas complemented with generated lightstimuli . Prototype 03 was shown at the expoSPOTLIGHTS at cc Hasselt in 2007 inBelgium.

Prototype 04 was developed between 2007and 2008 in close collaboration with theyoung Belgian curator Astrid David and thecollective Office Tamuraj . The music-generating engine was renewed whileconstructing this prototype. Musicgeneration in this new prototype was mainlybased on an automatic mixing engine.Under the supervision and lead of AstridDavid experiments were also being carriedout with different forms of presentationand confrontations between the EMO-Synthand other artistic disciplines. This resultedin the so-called test-cases. Testcase 1 washeld at the artist’s office in 2007 in Ghent,Testcase 2 was hosted by art spaceCroxhapox (Ghent, Belgium, 2007), Testcase3 was realised in collaboration with artorganization X=10c and was held at the artfair Lineart 2007 (Expo Hallen, Ghent,Belgium, 2007). Testcase 4 was held both atart gallery One Twenty Gallery (Ghent,Belgium) and at Muhka Media (Antwerp,Belgium) featuring the renowned Belgianpsychotherapist Wilfried Van Craen.

06

Prototype 05 is the current prototype of theEMO-Synth and has been underdevelopment since 2009. The creation of thisprototype is being supported by the FlemishAudiovisual fund in the seminal projectentitled “Montage cinema revived by the EMO-Synth”. This project can be seen as a logicalsequel on the different test cases heldbetween 2007 and 2008. As to the technicalside Prototype 05 includes a renewed soundgeneration engine which incorporates allprevious technologies and a firstimplementation of an image generationsystem.

07

The general idea behind the EMO-Synth isto create a dynamic work of art that evolvesautomatically. Using a natural evolutionaryprocess the computer will pursue thecreation of ideal optimized and personalaudiovisual material. The EMO-Synth thusembraces a fasciniting new paradigm thatopens the discussion on the deprecation ofthe unique character of a piece of art.Referring to earlier art history; this is arecurrent discussion that also appeared atthe rise of photography at the start of the20th century. At that time the generalconcern was the loss of the aura thatsurrounds artwork because reproduction ofphotographic material severely questionedartistic authenticity. But as history thoughtus it turned out artistic potential is notequal to its medium.

Montage cinema revived by the EMO-Synth is aproject based around the EMO-Synth whereexperimentation with the visual applicationof the EMO-Synth is a central key point. Theproject was a result of a series ofperformances so called test cases – whichOffice Tamuraj elaborated between 2007and 2008 and where I acted as curator. Inthis respect “Montage Cinema Revived by theEMO-Synth” could be seen as a project inwhich the experience and techniques usedin these testcases are merged. Themesexplored in testcases 1 to 4 includedautomatic music and image generation, theuse of color- and light dynamics and

psychophysiology. During this process as acurator I safeguarded the audiovisual resultfor which the film constitutes the apexHaving a background in art sciences Ilooked for ways to promote the thesis thatthe EMO-Synth project belongs to thedomain of audiovisual arts. In order to getthis statement in the public artistic debateI organized dialogues and reflectivemoments.

In my point of view the EMO-Synthperformances refer to developments in theFluxus movement of the 70s. The fact thatValery Vermeulen visualizes heart ratefrequencies and statistical data shows a lotof similarities to how Bernar Venet puttogether very explicit seance-performanceswith mathematicians and scientists from1966 to 2006. Bernar Venet moved fromNice to New York where he joined a groupof people with backgrounds ranging fromnuclear physics to math and music. In thisrespect Office Tamuraj can identify itself tothe early group of inventors of multimediaand the pioneers of video-art with JohnCage, Nam June Paik, Yoko Ono and BruceNauman as leading figures. Among thesethe work of Nam Jun Paik seems to be mostrelated to Vermeulen’s artistic approach.Like Nam Jun Paik, Valery Vermeulen has thetalent to renew the arts on a technical,visual and auditory level. The artist ValeryVermeulen continuously switches betweenthree domains by applying scientific

PROTOTYPE 05

Montage cinema revived by the EMO-Synth

Introduction

08

research to his creative process. Heconsciously puts pure mathematics in thebackground to serve the artistic result.

As the development of the EMO-Synthproject has an outspoken interdisciplinarycharacter the collective Office Tamurajwas founded to elaborate the variousaspects of the project. Along the waythese collaborations seemed to be seminalfor the creation of several prototypes. Theaddition of programmer Stif Labarqueturned out crucial for a high endimplementation of the current prototype,the editing of the film by Arpad Dorozmaiwas essential to be able to talk of montagecinema and the collaboration with LiardKranen helped to bring the EMO-Synth towhat it is today. In collaborating withValery I ’ve always pursued my ambition torealize a work in team setting where eachmember plays his own crucial role. OfficeTamuraj has tried to conquer new terrainin the audiovisual arts with its computercontrolled work of art. The drive to

capture the human emotion by means ofmusic is perhaps romantic, but it couldlead to new to new artistic disciplineswhere the visitor can engage in a directand concrete relationship. The purelyobservational vanishes in favour of a realinteractive component.

Astrid DavidFreelance Curator

I f you want to bu i l d a sh i p , don ’ t

d rum up the men to gather wood ,

d iv i de the work and g ive order s .

I n s tead , teach them to yearn for

the vas t and end l e s s sea .

An to i ne de Sa i n t Exupery

09

The concept of montage cinema wasconceived by a number of Russianfilmmakers in the 1920s and its core ideaswere rooted in the Soviet revolution a coupleof years prior. Different fi lmmakers, such asVertov, Pudovkin and Kuleshov, all had quitedifferent ideas about what montage cinemaought to be. Therefore the word “montage”,which literally means editing, is used todescribe the Russian cinema of the 1920s. Inthis project though, we started off with thetheories and views of Sergej Eisenstein whoclaimed that montage was “the very essenceof cinema”.

His idea of “montage cinema” was thatfi lmmakers needed to compose a series ofexciting events in order to emotionallystimulate the viewer. This meant thatindividual shots couldn’t just follow eachother, but had to collide. He denounced theclassical Hollywood way of editing in whichtemporal and spatial continuity werequintessential. Shots had to be opposed toone another instead in a ‘dialectical way’. Thisidea was derived from the Marxist theory inwhich thesis and anti-thesis combined intosyn-thesis and thus determined the courseof history. This collision of shots could bebased on conflicts of scale, volume, motionand speed, both within and betweendifferent frames. These conflicts result in anew mental concept which forms itself insidethe viewer’s mind. This mental conceptcannot originate from the different shots co-existing, but emerges from the overallinteraction between shots. This enabled thedirector to raise abstract concepts, such asspecific emotions, in the mind of theaudience. Genuine emotions could not be felt

through emphatic identification with anactor/actress playing a certain role andacting out an emotion. True emotions couldonly be felt by the viewer through thedialectical process of images colliding andthereby generating meaning, supersedingthe sum of its parts.

A frequently used example to explain thistheory of how cinema works is the famousscene from Eisenstein’s fi lm Strike in whichwe see workers who are on strike and thenget attacked by law enforcement agencies.Subsequently we see bulls who are guidedtowards the slaughter-house. Thejuxtaposition of these shots creates ametaphor in the viewers mind about theworkers being treated as cattle and stirs uprage and discord about the way the workersare humiliated. The metaphor in this exampleacts as a catalyst of emotions that could nothave been set off by merely showing imagesof the workers and the reaction of theestablishment to their uprising against thesystem.

Montage cinema revived by the EMO-Synth

project outline

1 0

PROTOTYPE 05

1 1

“Montage cinema revived by the EMO-Synth” ismainly focused on the development ofPrototype 05 of the EMO-Synth incollaboration with the collective OfficeTamuraj . Right from the start of thedevelopment of the EMO-Synth there werealready some ideas to use it in a moviecontext. After the initial developments of thefirst four prototypes funds provided by theFlemish government (supported by theFlemish Audiovisual Fund, www.vaf.be) nowmake such a realisation possible.

The main goal of “Montage cinema revived bythe EMO-Synth” is to explore and implementmodules for the EMO-Synth so that it can beused in a cinematographic setting. Thegeneral idea – in line with the general ideasand philosophy of the project – is to radicallyabandon the concept of a static movie.Instead we programmed the EMO-Synth togenerate new versions of any given moviewith the appropriate, personalizedsoundtrack. The audiovisual material isgenerated in a way that maximizes theemotional impact on the user.

For Prototype 05 images were needed thatcould provoke strong emotional statesduring the performances. To meet thisrequirement we decided to work with an oldclassic si lent movie. After some internaldebate, Office Tamuraj chose Julian Ruperts1925 version of The phantom of the Opera tobe the footage we would work with.

“Montage cinema revivedbythe EMO-Synth”alsodenotes the implementation of our firstimage generating system. It is basically anautomatic video montage system in thiscase. During performances the EMO-Synthwill automatically start producing a newmontage of the given movie, whilepreserving semantic content. Implementingthis first image generating system hasalready proved to be crucial to the furtherevolution of the image-generatingcomponent of the EMO-Synth.

“Montage Cinema Revivedbythe EMO-Synth” is abrand-new interactive cinema experiencethat meets the expectations of the vieweras closely as possible.

PROTOTYPE 05

Montage cinema revived by the EMO-Synth

The Phantom Of The Opera

1 2

The Phantom of the Opera is a 1925 film directed by Rupert Julian. The film is about Eric, amusical genius who hides himself because of his deformed face. After he has heardChristine, the young opera singer, he becomes hopelessly in love with her. This is thebeginning of a tragedy. Christine is engaged to the handsome young count Raoul de Chagnyand cannot feel love for a deformed freak. Driven by extreme anger and jealousy thephantom kidnaps the young soprano.

As most of the work of the montage cinema school is protected by legal issues we lookedin the project for alternative film material apt to being applied in the "montage cinema"context. Extensive research lead us to The Phantom of the Opera. Even though it was madein roughly the same period as the montage film-school flourished, it has few similarities atfirst glance. But closer examination reveals that it contains all the elements to be used formontage purposes.

On one hand there is the world of the phantom, which is dark, gloomy and driven by negativeemotions. On the other there is Christine’s world which is bright, happy and has overallpositive connotations of love and devotion. The emotional elements within the shots areone-dimensional and don’t really evoke complex emotions and abstract concepts in theviewer’s minds. But a clever re-editing, executed by the EMO-Synth which follows theguidelines stipulated by the director of the project, adds a new dimension to the film.

The juxtaposition of shots of both “worlds” will lead to the emergence of new and morecomplex and abstract emotions which will be reinforced by the new soundtrack that iscomposed to the slightly re-edited parts of the film. The audience will experience empathyfor the villainous character of Eric. This empathy is usually not associated to maliciouscharacters in fi lms and can only be achieved by means of using montage techniques. Theseelements present in The Phantom of the Opera make it the ideal fi lm to use in a montagecontext without being a montage film itself.

1 3

Because the elaboration of the different aspects of the EMO-Synth requires variousskills the dynamic collective Office Tamuraj was founded in 2007. The collective includespeople from different backgrounds including programmers, musicians, designers andpeople with an artistic aspiration. Our main philosophy as a dynamic collective is toallow people to join in or drop out freely. Being a collective also implies we want to voicea single message: Office Tamuraj aims to translate recent developments in affectivecomputing to the world of music and image generation. In doing so it acts as a bridgebetween the world of purely academic research and the artistic world. Of course weare aware that we are not the only players trying to attain this goal. But in gatheringpeople with a common spirit and state of mind we want to establish a discussionplatform for the possibilities of creating a new era where man and machine cancommunicate on an emotional and artistic level. For the creation of “Montage CinemaRevivedbythe EMO-Synth”Office Tamuraj consists of the following people:

Valery Vermeulen – Electronic musician, composer, producer and developer of newinteractive multimedia software and founder of the EMO-Synth project. He obtained aPhD in mathematics at the University of Ghent where he conducted fundamentalresearch in the area of algebraic group theory and geometry (PhD in Pure Mathematicsin 2001). Between 2001 and 2005 Vermeulen worked at the Institute for Psychoacousticsand Electronic Music (IPEM, University of Ghent) on a research project focusing on thelink between music and emotions. Meanwhile he started writing and recording musicin his own production studio. Since 2004 Vermeulen has been working on thedevelopment of the EMO-Synth and other art projects focusing on sound and music.Besides being involved in the EMO-Synth project Vermeulen works currently as astatistical expert and consultant and studies music at the Royal Conservatory of Ghent.

Astrid David – Astrid David – Studied social-cultural work at the Social College Ghent,art-sciences at the University of Ghent, followed by an internship at the MetropolitanUniversity of Manchester. She worked as cultural policy coordinator on two acceptedcultural policy plans for Tielt and Eeklo, organized the exhibition Persévérance in the“Godshuis” of Sint-Laureins in 2006 and was associate at the André Simoens Galleryfrom 2006 to 2009. More info on www.david-vzw.be.

Arpad Dorozmai – Graduated in Communication Sciences from the Catholic Universityof Leuven in 2004 with a dissertation entitled “A filmtheoretical research on the social-criticism in Elia Kazan’s fi lms”. In 2004 he entered a Master-class in Film Studies andVisual Culture at the University of Antwerp. The following year he produced a shortdocumentary film entitled “Kandarock". For his master degree, he worked on a shortfi lm entitled “Nowhere to hide”. The subsequent two years he worked for the “EuropeanFilm Festival in Brussels”. He came into contact with Valery Vermeulen at Testcase 1 ofPrototype 04 and eventually became a member of Office Tamuraj .

Office Tamuraj

1 4

Liard Kranen – After graduating from Waldorf Education with a project and a paper on‘DOGMA95’ Liard enrolled in Economy and Arts at the Utrecht School of the Arts. Hegraduated in 2006 with honors. Liard was involved in numerous projects as a producersuch as the short fiction film ‘Hotel Paradi js’ from Jan Krüger and the long-play fiction‘De Legendarische Scheiding’ from Lars Trim and the documentaries ‘Hooligans’ fromNikita Sutyrin and ‘Kresty’ from Allard Detiger. After his these experiences in fi lm Liardobtained a Master of Arts degree in the EMMA European Media Master of Arts programof the HKU. In meantime he worked as a project manager for the Changing Societiesproject of BUZA (Dutch ministry of Foreign Affairs) the HKU and VGIK (All-Russian StateInstitute of Cinematography Moscow). His graduation project was called “Disnified”(www.disnified.org) an online interactive documentary film about the“entertainmentization” of society and the thoughts of young Europeans on this topic.Liard became one of the members of Office Tamuraj art collective in 2007. FurthermoreLiard is working on a photographic documentary and an audiovisual installation fromthe Disnified project.

Stif Labarque – In 2004 Stif Labarque obtained the degree of Industrial Engineerspecialized in Information Technology. For his thesis he wrote an application that servedas an online data loader to provide access analysis for a video streaming website forLa nueva España (a Spanish newspaper). Since then he has been developing severalweb-based and standalone applications using Microsoft’s .NET technology. He wascontacted by Valery for the development and implementation of Prototype 05 of theEMO-Synth. Through the use of cutting edge technologies he has seamlessly integratedthe new EMO-Synth’s core and it’s subsystems into one application.

1 5

Valery Astrid Arpad Liard Stif

1 6

1 7

DVD not here? Go to the med ia se c t i on at www.emo-synth . com

www.emo-synth.com

v.u.: Valery Vermeulen - OfficeTamuraj@gmail.com

design: www.jon-ibe.be

Flemish Audiovisual Fundwww.vaf.be

Flanders Imagewww.flandersimage.com

Mindmedia B.V.www.mindmedia.nl

Montage cinema revived by the EMO-Synth

was produced by

Montage cinema revived by the EMO-Synth

was supported by

Office Tamurajwww.myspace.com/OfficeTamuraj

More info about the EMO-Synth