- three continent networked music and dance performance - Barcelona(ES)-Salvador(BR)-ChiangMai(TH)-Daejeon(KR)

Draft Ver. 0.9 Feb 13th, 2012

APAN e-Culture WG

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 2

Introduction • During the APAN 2012 congress, there will be a Live Music and Dance

performance with interactive participation of people and organizations in three different continents / four sites: Chiang Mai/Thailand, Barcelona/Spain, Salvador/Brazil and Daejeon/Korea. The full shot of the main stage at Chiang Mai will be streamed to KISTI/Korea in real time.

Date: Monday, 13-Feb-2012 Time: 17:00 (Chiang Mai) | 11:00 (Barcelona) | 8:00 (Salvador) | 19:00 (Daejeon) Venue: APAN 2012 Main auditorium (Same as the APAN 2012 Opening Plenary)

Information

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 3

Performance Infrastructure • As the performance includes live interaction of dancers and audience in the four venues (endpoints) - Chiang Mai, Barcelona, Salvador and Daejeon - the network infrastructure must support the video applications requirements for live transmission of HD audio and video streams.

Monitoring • All endpoints must have in their connections monitoring of their respective network performance and monitoring of the multimedia applications.

Network Endpoints • Barcelona : Konic Thtr / i2cat • Chiang Mai : APAN main auditorium • Salvador : Technological Poetics/UFBA, LAViD/UFPB, RNP PoP-BA, RNP PoP-RJ • Daejeon : KISTI

Arthron (Linux)

Max6 (Windows)

Audio

Video Audio Network

self-view (Br)

Max6 (Windows)

Audio

Ba+Br+CM (composed)

HD-SDI

Audio/Video Setup

Barcelona

Salvador

Chiang Mai

HS400A

NETWORKS

View from Ba

self-view (Ba)

View from Br

Arthron (Linux)

HDMI HDMI

DVI DVI

HD-SDI DVI DVI

Arthron (Linux)

Arthron (Linux)

Max6 (Windows)

Audio

Full shot of CM Full shot of CM

1394 Arthron (Linux)

Full shot of CM

Daejeon

DVI

DVI DVI

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 5

Arthron 1 : Ubuntu 11.04 OS CPU: Intel Core2 Duo 2.13GHz RAM: 4GB (3.16GB) VGA: Quadro FX 1500 256MB (2 DVI, 1 S-Video) Black Magic Studio IEEE 1394 Arthron 2 : Ubuntu 11.04 OS CPU: Intel Xeon 5110 1.6GHz RAM: 4GB (3.16GB) VGA: ATI Radeon HD 5570 1GB (1 DVI, 1 HDMI, 1 S-Video) Black Magic Studio IEEE 1394 Max 6 : Windows XP CPU: Intel Xeon X3330 2.66GHz RAM: 4GB (3.62GB) VGA: on broad Black Magic Duo IEEE 1394 Video Mixer: AV-HS400AE 2 DVI-IN, 4 SDI-IN, 4 SDI-OUT Camera : Canon XHG1

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 6

KISTI|Daejeon/KR KAIST|Daejeon/KR

APAN|Chiang Mai/TH

Bangkok

Hongkong

Seattle Chicago

Miami

Network Path

São Paulo

Amsterdam

i2cat/Citilab/|Barcelona/ES KONIC Thtr|Barcelona/ES

RNP|Sao Paulo/BR Tech Poetics/UFBA/BR

1G

Minimum Bandwidth requirement: 100Mbps

APAN Venue

10G (+600Mbps)

ChiangMai University

JGN-X Hong Kong

HDPMnet : LP (L2)

Network Topology Diagram 1G

KAIST 10.0.1.85~86 (test) 134.75.214.21~30

10.0.1.30~31 (test) 134.75.214.31~40

10.0.1.165~167, 169(test) 134.75.214.41~50

Designed by Buseung Cho Modified at Feb 7th 2012

VLAN 2780

HK-IX

ThaiREN Bangkok, TH ThaiSARN

Bangkok, TH

UniNet Chiangmai, TH

SouthernLight / Sao Paulo, BR

NetherLight, Amsterdam, NL

Starlight Chicago, US

134.75.214.11~20

134.75.214.1 (GW)

622Mbps

AMPATH/ Miami (FL), US

KRLight Seattle PoP, US

KRLight Daejeon PoP, KR

KRLight Hong Kong PoP, CN

AmLight East

KRLight Hong Kong

1G

Bangkok – Singapore - Hong Kong

134.75.214.11

MAPA-D2, UFBA (Fed. Uni. of Bahia)

KISTI

VLAN 2781

Kònic

Barcelona, Spain

Daejeon, Korea

Salvador, Brazil

Chiangmai, Thailand

C-wave

RNP backbone Brazil

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 8

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 9

Network Infrastructure

Endpoints Test IP Real IP

Barcelona , Spain Kònic 10.0.1.30 ~ 31 134.75.214.31 ~ 40

Chiang Mai, Thailand APAN Venue 202.28.194.4 134.75.214.11 ~ 20 134.75.214.11 : Network interface

Daejeon , Korea KAIST 10.0.1.85 ~ 86 134.75.214.21 ~ 30 134.75.214.1 : Gateway (Hong Kong) KISTI 10.0.1.80 - 10.0.1.89

Salvador, Brasil Ivani Lab

10.0.1.162 = MAX6 (A/V host) 10.0.1.163 = Encoder (A/V Camera) 10.0.1.165 = Articulador (A/V Flux mgr.) 10.0.1.166 = VC equip. 10.0.1.167 = Ivani's notebook (Test host) 10.0.1.168 = Switch ASGA

134.75.214.41 ~ 50

IP range: 10.0.1.160 –

10.0.1.169/24 Multicast:

233.3.50.160 – 233.3.50.169

LAViD Lab 10.0.1.169 = Test / A/V Host

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 10

Software : Arthron

ARTHRON was developed to assist the execution of artistic performances that make use of multimedia representations and real time sharing of real and virtual spaces. In this context, Arthron was used in the domain of research and development in Art and Technology, as well as in artistic presentations like Versus (2005), (In)TOQue (2008) and e-Pormundos Afeto (2009-1011) events. ARTHRON’s main functionality is the simple and direct interface, designed for the manipulation of different sources and streams of media (audio and video) simultaneously. This way the user can remotely add, remove and configure the presentation format as well as schedule the streams of media in time and space during an event. ARTHRON is composed of six main components: Articulator, Encoder, Decoder, Reflector, VideoServer and MapManager. The Articulator is responsible for the remote management of the other components, concentrating a great part of the ARTHRON functionalities such as stream scheduling (manual or automatic, via the creations of different scenarios), network monitoring and measurement, remote configuration of other modules, access control, automatic generation of a web page for online publication of low definition videos, addition of several video effects, chat for communication between the components and much more. The Encoder is responsible for the capture and encoding (when necessary) of the media source, which can be external (DV or HDV camera, DVD, etc) or internal (a local file). The Decoder’s main functionality is to decode and display the media stream in a specific device (monitor, projector, etc). The Reflector is responsible for the replication and redistribution of a media stream over the network. The VideoServer is able to transcode the media streams that will be published online, being able to transcode to the flv, ogg and h264 formats. The MapManager controls and displays the interactive map of the ARTHRON components. All streams can be generated in geographically distributed locations.

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 11

Fig. A : Arthron’s Articulator main screen Fig. B : Elements of the e-Pormundos Afeto (2009) event, on the stage of the Dragão do Mar Theater

In Figure B we can see a general overview of the objects in scene during the show. The dancer located inside the Dragão do Mar Theater, in Fortaleza/Brazil, can be seen in (1). The central screen that receives the video stream of the dancer from Barcelona/Spain can be seen in (3). On the left screen(2) is shown the video stream generated by the robot Galateia, from the NatalNet laboratory. On the right screen (4) is the video sent to the internet and several people could watch online, inside the GTMDA website. In (5) we can see the musician located in Fortaleza.

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 12

System module where the user can visualize the states (scenarios) and configure events

- Automatically generated web page of the transmission of a specific Decoder

“Versus” (2005) e_ePormundos Afeto (2009 - 2011) (In)TOQue (2008) By Ivani Santana By Ivani Santana & Konic Thtr

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 13

Cyber World, now the tools of the digital age give us to easily communicate with people on the other side of the world in new ways. New contacts create a new culture, new arts create new paradigm. Here, there are gestures from three locations meeting virtually and they are suddenly transferred to another space. Three points disperse and flock dreamily. Harbor your futuristic dream in the wide world while taking a cyberwalk with them.

(1’37”)

0’00” 0’40” 1’18” 2’55” 4’12” 4’42” 5’29” 6’39” 8’29” 10’27” (40”) (38”) (1’17”) (30”) (37”) (1’10”) (1’50”) (1’58”)

Intro Lure I Mystery Encounter

Lure II Longing Dream Separation Jealousy

Dan

ce Ba

Br

CM

Video blending delay + blending

mosaic

The Plot of the music

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 14

tempos moods places

40" Intro

38" Lure I

1' 37" Mystery

1' 17" Enconter (encontro)

30" Lure II

(atrair 2)

37" Jealousy (ciúme)

1'10" Longing (desejo)

1'50" Dream (sonho)

1'58" Separation (separação)

SPAIN

Arms and hands choregraphy. Medium shot

Close- up Hands Mosaic body fragments. CLose-up

Evolution Dance in fragments + signal Chiang Mai -Mosaic 3D

DancerTorso and spine + image deformation (3d convol.) Medium shot

dancer walking to the endless path. Long shot

THAILAND

Contemporary Dance

Thai dance & Modern dance

BRAZIL

abstract image from a part of the body and/or part of the face (part of the mouth) talking (without sound) = Camera shot: close up

the hand touching the other part of the body = Camera medium shot

dancer stands with his back to camera and with just slow head movement (video scenary behind him) = Camera full shot

the camera will capture the dancer image projected in a glass with water (illusion effect) = Camera shot: close up

dancer walking to the endless path (video scenary behind him) = Camera long shot

MIXER VIDEO blending mosaic delay +

blending

0’00” 0’40” 1’18” 2’55” 4’12” 4’42” 5’29” 6’39” 8’29” 10’27”

TECHNOLOGICAL POETICS RESEARCH GROUP & KONIC THTR & THAILAND GROUP

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 15

Remote Control for Music to synchronize audio and video from multi-sites

step 1 : measuring of RTT using OSC(Open Sound Control) • sending a 'bang' signal from CM to Ba/Br • The arrived 'bang' signal returns to CM immediately. • measuring the RTT between started 'bang' and returned 'bang' at CM (fig.1) ※ It takes about 380ms like ping.

fig.1 RTT measuring

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 16

step 2 : measuring of delay time including latency through ARTHRON from each site Ba/Br to CM

• shooting the Max6 window with a camera connected to ARTHRON at Ba/Br (fig.2) • sending a 'bang' signal from CM to Ba/Br, at the same time, started the video recording with a webcam

connected to Max6 machine at CM (fig.3) • When a 'bang' signal arrived at Ba/Br, the object in the Max6 window will blink in red. • The video recording at CM will continue until the red blinking on the video from Ba/Br is appeares • calculating video frames from the start to red blinking, and then multiply 33(FPS: 30) • The actual delay time is {video delay time - (OSC RTT/2)} ※ I predict it will take around 1~2s according to codec.

Arthron (Linux)

HDMI

to CM

fig.2 shooting Max6 window

fig.3 recording arrived video from far-end site

Remote Control for Music to synchronize audio and video from multi-sites(cont’)

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 17

step 3 : remote control of starting point of music at Ba/Br

• adjusting the delay time of each start signal sent to Ba/Br and of start signal for the local music which should be synchronized with receiving video from Ba and Br

• Finally, we can get synchronized audio and video all together between three sites.

CM

‘bang’ signal

delay 0ms

delay 700ms

starting music by remote control

160ms 140ms

starting music by remote control

Arthron (Linux)

HDMI Arthron (Linux)

HDMI

HS400A

Arthron (Linux)

Arthron (Linux)

delay 2,760ms

1,900ms

2,620ms

0+140+2,620 = 2,760ms

700+160+1,900 = 2,760ms

2,760ms

BC BR

The idea of synchronizing the synthesized video data with the prepared local sound at Chiang Mai will be able to provide more reliable quality of music. Also, 3D-speaker system can be employed on the procedure if it needs.

included latency

included latency

Remote Control for Music to synchronize audio and video from multi-sites(cont’)

one-way delay time one-way delay time

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 18

Organizer • Faridah Noor Mohd. Noor | um/my Chair of APAN e-Culture WG http://apan.net/wg/eculture.php

Steering • Dae Young Kim | cnu/kr • Artur Serra | i2cat/es • Michael Stanton | rnp/br • Panjai Tantatsanawong | uninet/th

Support • Patama | uninet/th, sec • Jeffrey Withaya Campbell | culture360/tw, web • Wichan Lertwipatrakul, uninet/th • Luiz Claudio Mendonça | cpd-ufba/br • George Jales | cpd-ufba/br

Artist • BonCheol Goo | kaist/kr • Rosa Sánchez | kònic thtr/es • Alain Baumann | kònic thtr/es • Sachiko Fullita | kònic thtr/es • Ivani Santana | technological poetics/br • Aldren Lincoln| technological poetics/br • Hugo Leonardo| technological poetics/br

Audio/Video • Andrew Howard | anu/au • Seongtaek Lim | kaist/kr • Fco. Javier Iglesias Gracia | i2cat/IGLOR/es • Erick Melo | LAViD/br • Sindolfo | LAViD/br • Felipe André| technological poetics/br • Bruno Rohde| technological poetics/br • Jacson Espirito Santo| technological poetics/br • Wutjanun M. (Nunny) | mu/th

Participants (PD/Artist)

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 19

• UniNET, Chiang Mai Venue (Thailand) –Phatcharapong, apan-noc@uni.net.th –Tawee, tawee@uni.net.th –Nunny, nugeog@hotmail.com

• ThaiSARN (Thailand), tsn-noc@nectec.or.th –Chalermpol, chalermpol@nectec.or.th

• i2Cat (Spain) –Sergi Figuerola, sergi.figuerola@i2cat.net –Vicent Borja, vicent.borja@i2cat.net –Artur Serra, artur.serra@i2cat.net –Pedro Lorente, pedro.lorente@iglor.es –Chalermpol, chalermpol@nectec.or.th

• StarLight (US) – Joe Mambretti, j-mambretti@northwestern.edu – Jim Hao Chen, jim-chen@northwestern.edu – Fei I Yeh, fyeh@northwestern.edu – Alan Verlo, darkman@evl.uic.edu •APAN-KR (Korea)

–Jaehwa Lee, jhlee@kr.apan.net (APAN-KR)

• RNP (Brasil) –Michael Stanton, michael@rnp.br –Iara Machado, iara@rnp.br –Leandro Ciuffo, leandro.ciuffo@rnp.br –Marcel Faria, marcel@nrp.br –Fausto Vetter, fausto.vetter@rnp.br –Alex Moura, alex@rnp.br

• APAN-JP (Japan) –Kitamura Yasuichi, kita@jp.apan.net

• JGN-X (Japan), noc@jgn-x.jp –Kazumasa Kobayashi , kazu-k@cs.kusa.ac.jp –Takatoshi Ikeda, ikeda@ote.kddi.com

•Cisco/C-Wave – Kevin McGrattan, kmcgratt@cisco.com

•UFL/FLR – Chris Griffin, cgriffin@ufl.edu

•FIU/AMPATH – James Grace, jgrac002@fiu.edu

•KISTI, KREONET/KREONet2/KRLight (Korea) – Buseung Cho, bscho@kisti.re.kr –MinKi, Noh, mknoh@kisti.re.kr

Participants (Network)

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 20

ORG Name Country Address

KAIST Korea Advanced Institute of Science and Technology Korea

291 Daehak-ro , Yuseong-gu, Daejeon 305-701, Korea +82-42-350-2114 - Tel. +82-42-350-2210 (2220) - Fax.

KISTI Korea Institute of Science and Technology Information Korea

245 Daehak-ro, Yuseong-gu, Daejeon. 305-806, Korea +82 42 869-0960 - Tel. +82 2 3299-6244 – Fax.

i2Cat Fundació privada i2CAT Internet i Innovació Digital a Catalunya Spain

C\ Gran Capità 2-4 Edifici Nexus I, 2ª planta, desp. 203 08034 Barcelona Tel. (+34) 93 553 25 10 Fax. (+34) 93 553 25 20

LAViD Laboratório de Aplicações de Video Digital (UFPB) Brazil

RNP Rede Nacional de Ensino e Pesquisa Brazil

Rua Lauro Müller, 116 Room 1103 Botafogo 22290-906 Rio de Janeiro, RJ +55 21 2102-9660 - Tel. +55 21 2279-3731 - Fax.

GP Tech Poetics

Technological Poetics Research Group University of Bahia (UFBA) Brazil

Campus Universitário de Ondina s/n Salvador, Bahia, Brasil +55 71 3283.6981

Thailand

Organizations

DQ12 Dancing across Oceans Feb 13th 2012 @ ECC, Thailand, e-Culture WG, 33th APAN meeting 21