Ego-Jections: A “Theatrical” Reactive System of Kinetic Structures for Augmented Multimedia Performance

7/31/2019 Ego-Jections: A “Theatrical” Reactive System of Kinetic Structures for Augmented Multimedia Performance

http://slidepdf.com/reader/full/ego-jections-a-theatrical-reactive-system-of-kinetic-structures-for 1/7



machine/robotic agents integrating multimedia objects totheir functionalities (metabolic processes).

II. D ESIGN PROCESS

The design process was based on two parallel procedures. One being an aesthetic approach to the actual

appearance and form of a particular machine agent, proposed by artist and Medea Electronique foundingmember Christos Laskaris, the other being a continuousdynamic collective process taking place in MedeaElectronique’s regular brainstorming meetings, with the

purpose of answering the questions mentioned above.

Christos came with the idea of a short of “wearable”installation, initially to be suspended from the ceiling,functioning as a ‘head projectile’ for human agents. Inthis way an intimate, peculiar, connection between theenvironment/space and the human user was established.

Figure 1. Initial drawing

Figure 2. Initial drawing evolved

The above photos show the initial ‘blueprints’. Thefinal form of the installation is very similar to these

proposals.

Having these initial ideas to work with it was obviousthat the form of the installation itself proposed a

modified, augmented relationship between the user andthe physical space in one level and the augmented

machine/man creature and the environment, including therest of the human agents being present, on another.

Given the fact that this augmentation was implemented by the technological extension/intervention, it wassensible to consider the relationship between human

beings in a sociological manner and the relationship eachhuman being develops (self image) towards its self,through technological advancements and means. The

projection of the latter in the social boundaries andinteractions, affects the processes taking place in theformer. Thus the name EgoJection was formed,suggesting a network of technological enhancedorganisms developing interactions inside specific artisticcontext.

This relationship made obvious the presence of possible emergent phenomena concerning thefunctionality of a sole machine itself. At this stage it wasalso realized that however complex the movement and

functionality of one man/machine entity would be,additional machine agents would have to be added, inorder to create a complex system enabling the researchteam to analyze and control emergent behaviors, thuscreating a unpredictable but still at some levelcontrollable, immersive environment.

Taking into account the above discussion two differentscenarios were discussed, these scenarios are describedalong with the current status of the first machine agentcreated for presentation at Electromedia Works(emw’08) festival, at the User Interaction chapter. Thenext chapter describes the actual structure and thetechnology used for its implementation.

III. H ARDWARE SETUP

This first machine agent is a simple, interactive,robotic structure, comprised from a sensing system, anactuating system and a control system, functioning inessence as an autonomous nervous system. The structureuses video, sound and movement (current version onedegree of freedom only), to express its own status and itsunderstanding of the surrounding environment status.

It displays its function through two different states,depending on whether the cybernetic loop is closed,

through the presence of a human agent at its main input position.

The main components of the structure are:

A. Sensing Subsystem

State control of the installation is achieved through processing the information of the sensing device in frontof the ‘cockpit’ of the machine. Presence or not of a

participant is detected and the desired control state isinitiated.

Sensing devices are also placed on the body of theinstallation giving it the ability of determine whether

there are potential users in the perimeter around it.



Figure 3. Sensors arrangment

In total 8 infrared sensors are used in the currentversion.

B. Control/communication Subsystem

Figure 4. Microcontroller board

Communication between the computer and all thedifferent sensors and actuators was implemented using aMake Controller board. The Make Controller (http://www.makingthings.com/products/KIT-MAKE-CTRL/ ) is built around an Atmel AVR familyarchitecture RISC Flash memory microcontroller, theAT91SAM7X256. The controller’s board providescontrol, computation, along with CAN and Ethernetcommunication.

Figure 5. Analog inputs digital outputs

The application board supports connectivity throughUSB, CAN interface and Ethernet. It provides 8 analoginputs, 8 digital outputs and 4 servo outputs. These inputand output ports are used to read values from the sensors

and control all the actuating devices.

The boards are connected on a local Ethernet network with a 2 GHz Intel double core MacBook, running MacOS X version 10.4.8, controlling the actuating devicesand processing sensorial inputs, a desktop pc runningWindows Xp controlling and synchronizing the state of the machine with the video output, and a macmini

controlling the sound parameters. This network of computers is used for parallel processing of informationand control of the different subsystems simultaneouslyand in real time. It is the global control of the installationand able to process information coming from a systemwith much bigger complexity.

C. Actuating Subsystem

The basic actuator of the machine is a 1Hp AC motor made by Bonfiglioli mounted on the main structure withhandmade steel/copper mechanical joints.

An AC motor was used to provide sufficient directionand speed control and still being able to handle heavyloads and provide smooth silent movement. Other technologies like servo or stepper motors were not usedsince accuracy of positioning was not of an issue for thissetup.

D. Sound sources

As sound sources 4 loudspeakers (passive) are usedtwo inside the main body of the machine and 2 outside.The pairs of speakers do not perform simultaneously, butin relation to the current state of the system. Thisarrangement is implemented using the Kyma sound

processing platform.

E. Electrical Panel

An electrical panel is used to convert PWM signalsand digital signals, from the board’s digital outputs, to adrive signal controlling an AC, 1Hp, Bonfiglioli motor.The main component is a Telemecaique Altivar 31inverter, processing the signals coming from theMakingThings board.

Figure 6. Adding components to the electrical panel the last minute isalways a drag

http://www.makingthings.com/products/KIT-MAKE-CTRL/






In general the installation’s control panel (electrical panel along with the microcontroller board) is a ‘multi-converter’ of OSC and MIDI messages to voltagescapable of operating different electrical, pneumatic,mechanical components (motors, solenoids, pneumatic

pistons etc.) and assuring electrical safety during the

process. F. Main Wooden Frame and Mechanical Joints

The body of the installation is made of wood and paper. Stainless steel joints have been used in order totransmit the motion of the actuator to the rest of thestructure. The front part of the installation is a projectionscreen, made from a PVC panel.

A wooden boxlike frame, 2700x1400x1200 mm, has been constructed to shape the structure and to support themounted projection and camera. Paper cartons have beenused to seal the frame to provide an immersive, solitaryspace for the main participant.

Figure 7. Wooden frame

Figure 8. Wooden frame sideways

The whole frame is supported by stainless steel leg-like structures and joined using steel and copper

components allowing it to move in the vertical direction.All the mechanics and frames were tailor made by Medea

Electronique members, using milling machines and metal process equipment.

Figure 9. Assembling the steel structure

Figure 10. Frame, steel joints, actuator

Figure 11. Dressed with paper carton



Figure 12. Stainless steel kinematic system

Figure 13. Ready for test

G. Video/Projection subsystemA video screen in the front of the construction, was

used to project live video input from the installation’s‘cockpit’, seen both from the user ‘inside’ the installationand those from the outside.

Figure 14. ‘Cockpit’ with mounted projector and camera

A web cam mounted on the frame and a video projector also mounted inside the structure, provided thefootage and allowed a short of communication to theenvironment of the ‘internal’ structure’s processes.

II.S OFTWARE ARCHITECTURE

A. Firmware

Interfaces between the sensors and actuators and themicrocontroller interface device (Making Controller)have been developed for the following systems:

1. Processing: Java based programming environmentfor real time animated graphics synthesis andinformation processing. http://www.processing.org

2. Kyma: Graphic programming environment for real-time sound synthesis and control.http://www.symbolicsound.com

3. VVVV: Real time modular/nodal programmingenvironment for video and animated graphics processing. http://www.vvvv.org

These interfaces are implemented on the basis of aRTOS (Real Time Operating System). This softwareactivates all the useful hardware components for computation and communication. The firmware is

programmed in C programming language and includesfiles to:

- Provide network functionality

- Provide basic operating system functions

- Provide OSC and USB functionality

Protocols and functions supported on the firmware are:

- TCP/IP, used to provide network operations over the internet. Both UDP and TCP are supported

- OSC, is used as an option to connect all thesubsystems of the board

- Controller library functions, giving access to thecontrollers subsystems, for example the CANinterface or PWM devices

- Application Board library functions, providingaccess to additional subsystems for hardwarecontrol. For example the AnalogIn() functions

help read any of the 8 inputs or the Motor()functions provide DC motor control.

The RTOS uploaded on the microcontroller communicates with Processing . Processing performs allthe parameter computations and controls the flow of information between the software and hardwarecomponents.

The Processing engine is also used to create a simplevirtual model of the structure and its sensing devices,

providing visual feedback in the virtual plane on thestatus of these devices and in effect the status of theinteraction between human and machine agents. This is

achieved by a simple 2d representation of the structureand sensing devices range, along with their state (activeor not), using a color code simple system to represent it

http://www.processing.org/

http://www.processing.org/



(red color equals active area equals presence in thatspace). This allows control of the machine’s behavior according to environmental input information processing.

Figure 15. Simple 2d virtual status representation diagram

A uniform protocol was used on the basis of UDP theOSC (Open Sound Control) protocol. This architecturegives a uniform “transparency” to the different softwareand hardware platforms.

OSC messages are exchanged throughout the networkscomponents and platforms allowing the parallel

processing of information and essentially unified control.Messages are exchanged from the microcontroller boardto Processing, from Processing to VVVV, from VVVV toKyma and the microcontroller board, controlling the stateof the installation, in a continuous mode.

III.I NTERACTION MODEL PROTOTYPE

The development goals were directed towards creatingan autonomous robotic agent, still able to perform andfunction as a subcomponent inside a superimposed morecomplex system if necessary. Since a system as such isyet to be implemented, the characteristics of a singleagent are going to be presented in this chapter.

A. Behavior:

The behavioral tactics of the machine-entity dependsmainly on the environmental information provided by thesensing devices all around it. There are two main states,including several sub-states in each one, reflecting the

basic behavior of the system towards its humancounterparts.

The first is an idle state, where there is no humanagent, completing the cybernetic loop with the machine,in front of its “drive” position. In this state the machine

performs ‘by itself’, inviting the participants to join. Itshows elements of “life” through periodical animation of its structure, always depending on the presence or not of human-agents around it. The essential information are

provided from the seven sensors attached to its perimeter.

Figure 16. Transition from active to idle state

In the second state the machine ”recognizes” an active participant in front of its predefined drive position, andstarts performing its routine, always affected from theinformation of human presence around it. Thisinformation, is given by the sensor attached at the‘cockpit’, which is the predefined drive position.

Multimedia themes are presented in accordance to themachine-man interaction status supporting the ongoing

performance.

Figure 17. Active state

Figure 18. Active state multimedia



Figure 19. Transitions between active substates

Figure 20. P.Tsangarakis programming vvvv

Assigning character to the machine agent is of coursean illusive attribute implemented through theconnection/relation between the agents and the

performance of each subsystem. It is the artificialorganism’s performance, utilized with machine motionand producing sound and light while functioning, thatgenerates the notion of behavioral structures.

The resultant behaviors in a system of machine/manorganisms as such, can vary, from peaceful to aggressive,linear to chaotic, attractive to repulsive, individual toflocking, according to the systems emergentcharacteristics.

IV.D ISCUSSION

Appropriate design and architecture, even in smallscale interaction systems can allow the creation of complex, meta-systems able to perform structured,identifiable behavioral strategies, through simpleinteractions between the system components.

Permitting and conditioning agents to operate inunison, allows the system to generate an integratedoutput. There are key points in architecture morphologyof such systems, to facilitate multi-agent interaction. Thefocus while designing these systems is not their appearance, but the flow and processing of information.

Conditioning the unified sub-systems enables us toexpress specific design and functional scenarios,including the pursuit of valid artistic purpose.

V.F UTURE WORK

Additional software development and investigation of communication protocols, allowing the remote exchangeof data (music, video) is required. The project presentedis the first in a series of installations employing the samestructural design elements able to form a larger scale,more complex systems presenting emergent propertiesand characteristics. Larger networks are going to enableus to explore swarm networking between remote places,connected through Internet, exchanging cultural andenvironmental elements from differentlocations/cities/countries.

For the completion of different installations theutilization of various OSC to voltage converters isnecessary, providing flexibility during the design process.

It is also considered essential the development of different sensor devices, for the implementation of different interaction scenarios. Diverse designs and

technologies allow the implementation of sensors,varying from noticeable, personal, wearable, (single agentmonitoring), to invisible global ones, for crowd behavior and environmental monitoring.

ACKNOWLEDGMENT

Many thanks to “Constrinox SA”, D.Giannoukakis andA. Giannoukakis for providing means and materials usedin the installation, D.Tomaras and G.Tsotsos, helping inthe realization of the installation with their experience inmechanical engineering and electronics.

Documents

Ego-Jections: A “Theatrical” Reactive System of Kinetic Structures for Augmented Multimedia Performance