Upload
penda
View
22
Download
2
Tags:
Embed Size (px)
DESCRIPTION
e-Science Collides with the Performing Arts. Dr R P Fletcher University of York. So what is this e-Science??. At the inception of the UK e-Science programme John Taylor said that: - PowerPoint PPT Presentation
Citation preview
e-Science Collides with thePerforming Arts
e-Science Collides with thePerforming Arts
Dr R P Fletcher
University of York
Dr R P Fletcher
University of York
Thursday, 20 July 2007 Digital Representations of Performing Arts 2
So what is this e-Science??So what is this e-Science??
At the inception of the UK e-Science programme John Taylor said that:
“e-Science is about global collaboration in key areas of science and the next generation of infrastructure that will enable it”
Another source says it is:“science increasingly done through distributed global collaborations enabled by the Internet, using very large data collections, terascale computing resources and high performance visualisation “
Malcolm Atkinson has said that:“e-Science is the development and exploitation of advanced informatics methods to support collaborative research in all disciplines “
At the inception of the UK e-Science programme John Taylor said that:
“e-Science is about global collaboration in key areas of science and the next generation of infrastructure that will enable it”
Another source says it is:“science increasingly done through distributed global collaborations enabled by the Internet, using very large data collections, terascale computing resources and high performance visualisation “
Malcolm Atkinson has said that:“e-Science is the development and exploitation of advanced informatics methods to support collaborative research in all disciplines “
Thursday, 20 July 2007 Digital Representations of Performing Arts 3
OVERVIEWOVERVIEW
Academic Background – am I a real scientist?
Involvement in Performing Arts – am I a fraud?
Involvement with technology – am I a technologist?
TechnologyBespoke hardware and software
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future – some ideas
Questions – whatever you want to throw at me!
Academic Background – am I a real scientist?
Involvement in Performing Arts – am I a fraud?
Involvement with technology – am I a technologist?
TechnologyBespoke hardware and software
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future – some ideas
Questions – whatever you want to throw at me!
Thursday, 20 July 2007 Digital Representations of Performing Arts 4
My Academic backgroundMy Academic background
1968 Degree in BiologyGeneral
1971 D.Phil in NeurophysiologyInformation processing in single sensory nerve cells
1975 Research Fellow in PsychologyRisk of schizophrenia
Psychophysiology
1981 Computing ServiceGraphics
Database systems
Programming
Mathematical and statistical systems and libraries
1968 Degree in BiologyGeneral
1971 D.Phil in NeurophysiologyInformation processing in single sensory nerve cells
1975 Research Fellow in PsychologyRisk of schizophrenia
Psychophysiology
1981 Computing ServiceGraphics
Database systems
Programming
Mathematical and statistical systems and libraries
Thursday, 20 July 2007 Digital Representations of Performing Arts 5
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Thursday, 20 July 2007 Digital Representations of Performing Arts 6
My “Performing Arts” backgroundMy “Performing Arts” background
1965 Staged school bands1968 Staged University bands1971 President of the Electronic Arts Society1972 Recording engineer York Musicians Cooperative1972 Worked with Trevor Wishart1973 Recording/mixing engineer campus band1974 Built own synthesiser system1976 Programmed FX on PDP-111978 Recording/mixing engineer campus band (again)1991 SGI Indigos bootstrap Music Technology1996 Chaotic Constructions1999 RIMM2005 UKLight for Collaborative Composition/Performance 2006 PEACHES committee of TERENA2007 Content Creation tools for Digital Cinema?
1965 Staged school bands1968 Staged University bands1971 President of the Electronic Arts Society1972 Recording engineer York Musicians Cooperative1972 Worked with Trevor Wishart1973 Recording/mixing engineer campus band1974 Built own synthesiser system1976 Programmed FX on PDP-111978 Recording/mixing engineer campus band (again)1991 SGI Indigos bootstrap Music Technology1996 Chaotic Constructions1999 RIMM2005 UKLight for Collaborative Composition/Performance 2006 PEACHES committee of TERENA2007 Content Creation tools for Digital Cinema?
Thursday, 20 July 2007 Digital Representations of Performing Arts 7
Technology Input and OutputTechnology Input and Output
Specialised software to edit synthetic speech
User interfaces for Music Technology
Courses in graphics for Music Technology
External Examiner MA Digital Arts (Middlesex)
Midi I/O systems for Sun Workstations
Developer for GIMP (video plugins for Unix)
Developer for XMX (collaborative X-Windows)
Developer for .ogg audio formats Still ambisonic developments
Tracking technology for Chaotic Constructions
Graphics system designer/developer for RIMMReal-time Interactive Multiple Media
PEACHES committee at TERENAPanel of Experts for Arts, Culture and Humanities E-Science
Specialised software to edit synthetic speech
User interfaces for Music Technology
Courses in graphics for Music Technology
External Examiner MA Digital Arts (Middlesex)
Midi I/O systems for Sun Workstations
Developer for GIMP (video plugins for Unix)
Developer for XMX (collaborative X-Windows)
Developer for .ogg audio formats Still ambisonic developments
Tracking technology for Chaotic Constructions
Graphics system designer/developer for RIMMReal-time Interactive Multiple Media
PEACHES committee at TERENAPanel of Experts for Arts, Culture and Humanities E-Science
Thursday, 20 July 2007 Digital Representations of Performing Arts 8
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Thursday, 20 July 2007 Digital Representations of Performing Arts 9
Technologies Technologies
Aim to lead you through some technologies
Ones I have worked on over the last 15 years (or so!)
At some point technology overlaps e-Science
Need to engage with the new technologies
Need to help drive their development
Performing Arts need new/different facilitieseither from the existing infgrastructures
or via new facilities
But, unless we know what’s possible
How do we know what to ask for?
Aim to lead you through some technologies
Ones I have worked on over the last 15 years (or so!)
At some point technology overlaps e-Science
Need to engage with the new technologies
Need to help drive their development
Performing Arts need new/different facilitieseither from the existing infgrastructures
or via new facilities
But, unless we know what’s possible
How do we know what to ask for?
Thursday, 20 July 2007 Digital Representations of Performing Arts 10
YORKTALKYORKTALK
Worked with the Language and Linguistics staff
Their technology allowed about six synthesis runs PER DAY!
Data transferred via Kermit file transfer program!
Real informed trial and error to fix problems
I created a graphical interface to the data
Ran on a then high performance system
Could interact with the data
Synthesis was up to near real time
Combined visualisation to “see” errors
Used interactive graphics to “fix” the problems
Worked with the Language and Linguistics staff
Their technology allowed about six synthesis runs PER DAY!
Data transferred via Kermit file transfer program!
Real informed trial and error to fix problems
I created a graphical interface to the data
Ran on a then high performance system
Could interact with the data
Synthesis was up to near real time
Combined visualisation to “see” errors
Used interactive graphics to “fix” the problems
Thursday, 20 July 2007 Digital Representations of Performing Arts 11
YORK TALKYORK TALK
Research vehicle for speech synthesis
DecTalk example (may recognise the voice)
YorkTalk examplesSame words as DecTalk
A sentence (with a cold!)
Research vehicle for speech synthesis
DecTalk example (may recognise the voice)
YorkTalk examplesSame words as DecTalk
A sentence (with a cold!)
Thursday, 20 July 2007 Digital Representations of Performing Arts 12
Engaging with “artistsEngaging with “artists
Next project was with staff in Music
Needed help to create interfaces for a system
Needed some lateral thinking
Definitely moving into the Arts domain
High performance computing was needed
Multiple tasks by same computerVideo
Audio
Graphics
Midi
Next project was with staff in Music
Needed help to create interfaces for a system
Needed some lateral thinking
Definitely moving into the Arts domain
High performance computing was needed
Multiple tasks by same computerVideo
Audio
Graphics
Midi
Thursday, 20 July 2007 Digital Representations of Performing Arts 13
Chaotic Constructions - 1994Chaotic Constructions - 1994
Certain amount of serendipity
Peter Fluck talking on Radio 4 about Chaotic motion in kinetic scuplture
Tony Myatt talked a few weeks later about Chaos in Music
Phoned each other classic “WE SHOULD TALK” moment
By 1996 developed a projectto use the chaotic motion of a kinetic sculpture to drive a music composition system
PROBLEM – how to capture the motion
ANSWER – ask me
Certain amount of serendipity
Peter Fluck talking on Radio 4 about Chaotic motion in kinetic scuplture
Tony Myatt talked a few weeks later about Chaos in Music
Phoned each other classic “WE SHOULD TALK” moment
By 1996 developed a projectto use the chaotic motion of a kinetic sculpture to drive a music composition system
PROBLEM – how to capture the motion
ANSWER – ask me
Thursday, 20 July 2007 Digital Representations of Performing Arts 14
Chaotic Constructions – the Tracking SystemChaotic Constructions – the Tracking System
Request came to meHow can we do this?and …It will be installed in Tate Modern in March 1997and …It was now January 1997!
First thoughts were to use shape tracking systemsProblems:
most systems were proprietary and expensiveshapes to be tracked changed because sculpture was 3 dimensionalcould not use trajectory prediction due to the chaotic motion
ANSWER – simply track unique coloured items
Request came to meHow can we do this?and …It will be installed in Tate Modern in March 1997and …It was now January 1997!
First thoughts were to use shape tracking systemsProblems:
most systems were proprietary and expensiveshapes to be tracked changed because sculpture was 3 dimensionalcould not use trajectory prediction due to the chaotic motion
ANSWER – simply track unique coloured items
Thursday, 20 July 2007 Digital Representations of Performing Arts 15
VIDEO EXAMPLEVIDEO EXAMPLE
The Making
of
CHAOTIC
CONSTRUCTIONS
The Making
of
CHAOTIC
CONSTRUCTIONS
Thursday, 20 July 2007 Digital Representations of Performing Arts 16
Stage 1 – Define a Colour “Fingerprint”Stage 1 – Define a Colour “Fingerprint”
Program creates a set of “unique fingerprints”Program creates a set of “unique fingerprints”
Thursday, 20 July 2007 Digital Representations of Performing Arts 17
Tuning the “Fingerprints”Tuning the “Fingerprints”
Check motion does not lose sync with colour Check motion does not lose sync with colour
Thursday, 20 July 2007 Digital Representations of Performing Arts 18
Peter Fluck Hand Cranking the SculpturePeter Fluck Hand Cranking the Sculpture
Thursday, 20 July 2007 Digital Representations of Performing Arts 19
Running with Tracking and Music ProgramRunning with Tracking and Music Program
“Fingerprint” used by second program
Motion tracking is just one element
“Fingerprint” used by second program
Motion tracking is just one element
Thursday, 20 July 2007 Digital Representations of Performing Arts 20
Changing a Tracking PointChanging a Tracking Point
What are the best points to track?
Due to chaotic motion some trial and error!
What are the best points to track?
Due to chaotic motion some trial and error!
Thursday, 20 July 2007 Digital Representations of Performing Arts 21
And finally …the prototypeAnd finally …the prototype
… we get there … a small excerpt … we get there … a small excerpt
Thursday, 20 July 2007 Digital Representations of Performing Arts 22
And the real thingAnd the real thing
Thursday, 20 July 2007 Digital Representations of Performing Arts 23
Credits and AwardsCredits and Awards
Exhibited to great acclaim in Tate Modern
Other UK Art Galleries
Travelled to Hong Kong and Corsica
Herald Angel Award by the Glasgow Herald for the best piece of visual art at the 1997 Edinburgh Festival
International Critics Award as part of the 1997 Demarco European Art Foundation exhibition
Exhibited to great acclaim in Tate Modern
Other UK Art Galleries
Travelled to Hong Kong and Corsica
Herald Angel Award by the Glasgow Herald for the best piece of visual art at the 1997 Edinburgh Festival
International Critics Award as part of the 1997 Demarco European Art Foundation exhibition
Thursday, 20 July 2007 Digital Representations of Performing Arts 24
Not really “Performing Arts” I hear you sayNot really “Performing Arts” I hear you say
In St. Ives staff gave children coloured “bats”
Switched off the sculpture
Children waved the bats around
And, of course, “made music”
Security was called once to “remove” a visitor
Would not leave at closing time
Transfixed by the whole thing
Some people came 2 or 3 times a week
For a “fix”
In St. Ives staff gave children coloured “bats”
Switched off the sculpture
Children waved the bats around
And, of course, “made music”
Security was called once to “remove” a visitor
Would not leave at closing time
Transfixed by the whole thing
Some people came 2 or 3 times a week
For a “fix”
Thursday, 20 July 2007 Digital Representations of Performing Arts 25
How can this Technology be used …How can this Technology be used …
… for Performing Arts
Mount a camera above a stage
Give actors coloured hats
Now you have their position on stage
Link their movement around the stage
To …
Orchestrate lighting changes
HotspotsMove to one place on stage and make a sound
Detect “collisions”
Allow actors to fully interact with their environment
… for Performing Arts
Mount a camera above a stage
Give actors coloured hats
Now you have their position on stage
Link their movement around the stage
To …
Orchestrate lighting changes
HotspotsMove to one place on stage and make a sound
Detect “collisions”
Allow actors to fully interact with their environment
Thursday, 20 July 2007 Digital Representations of Performing Arts 26
…or ……or …
Point the camera at a tropical fish tank
Let the fish make music
Point the camera at a tropical fish tank
Let the fish make music
Thursday, 20 July 2007 Digital Representations of Performing Arts 27
Bringing e-Science into the FrameBringing e-Science into the Frame
So far York work done on a single modest computer
Scale upTo multiple specialist computers
Networks
Wireless devices
Complex tracking
3D sound
Interactive graphics
All controlled by …
The PERFORMER
So far York work done on a single modest computer
Scale upTo multiple specialist computers
Networks
Wireless devices
Complex tracking
3D sound
Interactive graphics
All controlled by …
The PERFORMER
Thursday, 20 July 2007 Digital Representations of Performing Arts 28
Real Time Interactive Multiple Media - RIMMReal Time Interactive Multiple Media - RIMM
“Construction 3”
Realised by the Real-time Interactive Multiple Media Project
University of York Music Department (UK), IRCAM (France) and SIMPK (Germany)
European Commission Information Society Technologies Programme
Incorporates a live performer on soprano saxophone
Three-dimensional sculpture
Computer generated graphics
Ambisonic surround sound
“Construction 3”
Realised by the Real-time Interactive Multiple Media Project
University of York Music Department (UK), IRCAM (France) and SIMPK (Germany)
European Commission Information Society Technologies Programme
Incorporates a live performer on soprano saxophone
Three-dimensional sculpture
Computer generated graphics
Ambisonic surround sound
Thursday, 20 July 2007 Digital Representations of Performing Arts 29
construction 3 is a synergetic compositionconstruction 3 is a synergetic composition
The musical and graphical elements of the piece have been composed concurrently
Both the computer graphics and music are triggered by the saxophonists natural gestures
Through infra-red video tracking
Use i/r device and i/r filter on camera
Electronic sensors
Footswitches, accelerometers
Machine listening
Score following
Gesture recognition using neural networks
e.g. “throw” sound into the audience
The musical and graphical elements of the piece have been composed concurrently
Both the computer graphics and music are triggered by the saxophonists natural gestures
Through infra-red video tracking
Use i/r device and i/r filter on camera
Electronic sensors
Footswitches, accelerometers
Machine listening
Score following
Gesture recognition using neural networks
e.g. “throw” sound into the audience
Thursday, 20 July 2007 Digital Representations of Performing Arts 30
Block Diagram of SystemBlock Diagram of System
Thursday, 20 July 2007 Digital Representations of Performing Arts 31
System DiagramSystem Diagram
Thursday, 20 July 2007 Digital Representations of Performing Arts 32
The “Team”The “Team”
Lots of people needed to make this happenProject manager to keep us in lineThe Composer/Sound stage designerThe Graphic DesignerThe Graphics ProgrammerThe Audio ProgrammerThe Electronics technical wizardThe Local site organiserThe commercial suppliersAnd …
THE PERFORMER
Lots of people needed to make this happenProject manager to keep us in lineThe Composer/Sound stage designerThe Graphic DesignerThe Graphics ProgrammerThe Audio ProgrammerThe Electronics technical wizardThe Local site organiserThe commercial suppliersAnd …
THE PERFORMER
Thursday, 20 July 2007 Digital Representations of Performing Arts 33
Team PhotoTeam Photo
Thursday, 20 July 2007 Digital Representations of Performing Arts 34
Screens and PerformerScreens and Performer
Thursday, 20 July 2007 Digital Representations of Performing Arts 35
Examples of the RIMM Computer GraphicsExamples of the RIMM Computer Graphics
Thursday, 20 July 2007 Digital Representations of Performing Arts 36
Technology becomes e-ScienceTechnology becomes e-Science
Clearly a lot of technology to make it work
Some is bespoke electronics
Majority of tasks were programmed
The SGI Origin 20008 fast processors
Highly parallel
jMax
24-bit optical output for audio
High speed network interconnect to graphics computer
SGI O2 graphicsOptimised for OpenGL
Texture memory drawn from main memory
Clearly a lot of technology to make it work
Some is bespoke electronics
Majority of tasks were programmed
The SGI Origin 20008 fast processors
Highly parallel
jMax
24-bit optical output for audio
High speed network interconnect to graphics computer
SGI O2 graphicsOptimised for OpenGL
Texture memory drawn from main memory
Thursday, 20 July 2007 Digital Representations of Performing Arts 37
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Thursday, 20 July 2007 Digital Representations of Performing Arts 38
Technologies for AllTechnologies for All
Many of these technologies are availableMany of the ideas are now embodied in softwareMany are free
Pure Data – aka PD (with GEM add-in)Eyesweb
VJ-ing is growingMany VJ products around
Grid Computing is availableNot easily accessibleLacks some functionality for Performing Arts
Photonic networks are availableHigh speed links point to pointUp to 10 gb/s possible now1 gb/s across the globe
Many of these technologies are availableMany of the ideas are now embodied in softwareMany are free
Pure Data – aka PD (with GEM add-in)Eyesweb
VJ-ing is growingMany VJ products around
Grid Computing is availableNot easily accessibleLacks some functionality for Performing Arts
Photonic networks are availableHigh speed links point to pointUp to 10 gb/s possible now1 gb/s across the globe
Thursday, 20 July 2007 Digital Representations of Performing Arts 39
jMax, Pure Data/GEMjMax, Pure Data/GEM
Graphics Programming
Lots of modules
Connect up inputs and outputs
Uses the modular synthesiser metaphor
GEMGraphics Environment for Musicians
Link audio to graphics
Limited by modules unless you write your own
Graphics Programming
Lots of modules
Connect up inputs and outputs
Uses the modular synthesiser metaphor
GEMGraphics Environment for Musicians
Link audio to graphics
Limited by modules unless you write your own
Thursday, 20 July 2007 Digital Representations of Performing Arts 40
eyeswebeyesweb
Video programming with modules
Huge number of modules
Includes “man” tracking capability
Can link to rudimentary audio
But …
Can also link to PDusing Open Sound Control protocols - OSC
Video programming with modules
Huge number of modules
Includes “man” tracking capability
Can link to rudimentary audio
But …
Can also link to PDusing Open Sound Control protocols - OSC
Thursday, 20 July 2007 Digital Representations of Performing Arts 41
REACTABLEREACTABLE
Collaborative electronic music instrument
Tabletop tangible multi-touch interface
Several simultaneous performers
Share complete control over the instrumentby moving and rotating physical objects
on a luminous round table surface.
Objects represent components of a classic modular synthesizer
can create complex and dynamic sonic topologies
a kind of tangible modular synthesizer or
graspable flow-controlled programming language
Collaborative electronic music instrument
Tabletop tangible multi-touch interface
Several simultaneous performers
Share complete control over the instrumentby moving and rotating physical objects
on a luminous round table surface.
Objects represent components of a classic modular synthesizer
can create complex and dynamic sonic topologies
a kind of tangible modular synthesizer or
graspable flow-controlled programming language
Thursday, 20 July 2007 Digital Representations of Performing Arts 42
REACTABLEREACTABLE
Thursday, 20 July 2007 Digital Representations of Performing Arts 43
REACTABLE - demoREACTABLE - demo
Click on Image to go to site with Videos
[will open a browser]
Thursday, 20 July 2007 Digital Representations of Performing Arts 44
More Video Tracking and PerformanceMore Video Tracking and Performance
Messa di Voce
Tmema,Blonk,La Barbara
http://www.tmema.org/messa/messa.html
Messa di Voce
Tmema,Blonk,La Barbara
http://www.tmema.org/messa/messa.html
Thursday, 20 July 2007 Digital Representations of Performing Arts 45
Messa di Voce - demoMessa di Voce - demo
Click image to go to the Host site
[will try to play Quicktime in a browser]
Thursday, 20 July 2007 Digital Representations of Performing Arts 46
And just for fun … another demoAnd just for fun … another demo
Thursday, 20 July 2007 Digital Representations of Performing Arts 47
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Thursday, 20 July 2007 Digital Representations of Performing Arts 48
So what is this Grid Computing?So what is this Grid Computing?
The term originated in the early 1990’s
Metaphor for making computer power as easy to access as an electric power grid
Many years on and still mainly in the domain of “BIG SCIENCE”
Most Grids are run like 80’s batch style computingSubmit a job description to a scheduler
It decides when and where the job runs – more or less
This is a computational grid
Many computers and clusters are interconnected
Can be globally distributed via photonic networks
The term originated in the early 1990’s
Metaphor for making computer power as easy to access as an electric power grid
Many years on and still mainly in the domain of “BIG SCIENCE”
Most Grids are run like 80’s batch style computingSubmit a job description to a scheduler
It decides when and where the job runs – more or less
This is a computational grid
Many computers and clusters are interconnected
Can be globally distributed via photonic networks
Thursday, 20 July 2007 Digital Representations of Performing Arts 49
What is this Data Grid?What is this Data Grid?
A grid computing system that deals with data
The controlled sharing and management of large amounts of distributed data
These are often, but not always, combined with computational grid computing systems
Finding data is not easy
Accessing it is not easy
Mainly in the domain of “BIG SCIENCE”
Some shift to Arts and HumanitiesWe have lots of data too!
More later in context of National Grid Service
A grid computing system that deals with data
The controlled sharing and management of large amounts of distributed data
These are often, but not always, combined with computational grid computing systems
Finding data is not easy
Accessing it is not easy
Mainly in the domain of “BIG SCIENCE”
Some shift to Arts and HumanitiesWe have lots of data too!
More later in context of National Grid Service
Thursday, 20 July 2007 Digital Representations of Performing Arts 50
What are these Photonics NetworksWhat are these Photonics Networks
A worldwide network of optical fibres
Speeds are 1gbit/s to 40 gbit/s
Run by different organisationsUK - Janet
Europe - GEANT2
USA - StarLight
USA - National Lambda Rail
Canada – CANARIE
etc …
A connection is usually point-to-pointVery exclusive
Lots of bandwidth
Low latency and jitter (more of this later)
A worldwide network of optical fibres
Speeds are 1gbit/s to 40 gbit/s
Run by different organisationsUK - Janet
Europe - GEANT2
USA - StarLight
USA - National Lambda Rail
Canada – CANARIE
etc …
A connection is usually point-to-pointVery exclusive
Lots of bandwidth
Low latency and jitter (more of this later)
Thursday, 20 July 2007 Digital Representations of Performing Arts 51
What might I use such links?What might I use such links?
Many examples
Interactive danceUse HDTV rather then Lo-res AccessGrid technology
Collaborative audio and videoMaster classes are more commonplaceCollaborative tools for distributed, remote teams
Digital cinema (4k x 2k) is used for:Entertainment, media, art and cultureScience, medicine, education and researchand also …Military, intelligence, security and police
Many examples
Interactive danceUse HDTV rather then Lo-res AccessGrid technology
Collaborative audio and videoMaster classes are more commonplaceCollaborative tools for distributed, remote teams
Digital cinema (4k x 2k) is used for:Entertainment, media, art and cultureScience, medicine, education and researchand also …Military, intelligence, security and police
Thursday, 20 July 2007 Digital Representations of Performing Arts 52
Nagano Winter Olympics 1998Nagano Winter Olympics 1998
Massive endeavour with live networked music
Opening ceremony had a chorus in:Berlin
Cape Town
Beijing
New York
Sydney
Performed the final movement of Beethoven's 9th
Together with the full symphony orchestra in:Nagano Kenmin Cultural Centre Hall.
Also …
2000-person chorus in the Olympic stadium
Massive endeavour with live networked music
Opening ceremony had a chorus in:Berlin
Cape Town
Beijing
New York
Sydney
Performed the final movement of Beethoven's 9th
Together with the full symphony orchestra in:Nagano Kenmin Cultural Centre Hall.
Also …
2000-person chorus in the Olympic stadium
Thursday, 20 July 2007 Digital Representations of Performing Arts 53
Nagano Winter OlympicsNagano Winter Olympics
The final result was mixed and broadcast worldwide
This setup required:7 satellites
12 uplinks with encoders and decoders
Multiple-delay compensation
The longest delay was to and from BerlinAround 4 seconds!
Incredibly …
It all worked and the result was orchestras and choirs all in time
The final result was mixed and broadcast worldwide
This setup required:7 satellites
12 uplinks with encoders and decoders
Multiple-delay compensation
The longest delay was to and from BerlinAround 4 seconds!
Incredibly …
It all worked and the result was orchestras and choirs all in time
Thursday, 20 July 2007 Digital Representations of Performing Arts 54
Orchestra RehearsalsOrchestra Rehearsals
New World Symphony in Miami rehearsed pieces by French composers
The composers participated live from IRCAM in Paris
This transatlantic rehearsal enabled live exchanges between musicians in Miami and the composers in Paris
Via RENATER-4 in France, GEANT2 in Europe, and Internet2
These advanced research and education networks provided the bandwidth and quality of service to ensure high quality audio and video
New World Symphony in Miami rehearsed pieces by French composers
The composers participated live from IRCAM in Paris
This transatlantic rehearsal enabled live exchanges between musicians in Miami and the composers in Paris
Via RENATER-4 in France, GEANT2 in Europe, and Internet2
These advanced research and education networks provided the bandwidth and quality of service to ensure high quality audio and video
Thursday, 20 July 2007 Digital Representations of Performing Arts 55
Using High Speed Networks for ArtsUsing High Speed Networks for Arts
Cinegrid is a good Example
Cinegrid’s Mission Statement:
To build an interdisciplinary community that is focused on the research, development, and demonstration of networked collaborative tools to enable the production, use and exchange of very-high-quality digital media over photonic networks
See www.cinegrid.org for more details
Cinegrid is a good Example
Cinegrid’s Mission Statement:
To build an interdisciplinary community that is focused on the research, development, and demonstration of networked collaborative tools to enable the production, use and exchange of very-high-quality digital media over photonic networks
See www.cinegrid.org for more details
Thursday, 20 July 2007 Digital Representations of Performing Arts 56
CinegridCinegrid
Converging on the digitalFast networking
Access to shared instruments, computers, storage
Collaboration tools
Robust security for intellectual property
High visual quality
Greater speed
More distributed applications
and, importantly …
Next generation of trained professionals
Converging on the digitalFast networking
Access to shared instruments, computers, storage
Collaboration tools
Robust security for intellectual property
High visual quality
Greater speed
More distributed applications
and, importantly …
Next generation of trained professionals
Thursday, 20 July 2007 Digital Representations of Performing Arts 57
Cinegrid@AES October 25th 2006Cinegrid@AES October 25th 2006
This event was structured in four acts
Each demonstrating a different facet of the CineGrid philosophy of networked extreme media
In Act 1, a sequence of 4K “digital shorts” at 24 frames per second (fps), together with fully mixed synchronized audio, were pulled in real time from network-connected servers in Los Angeles and San Diego.
This event was structured in four acts
Each demonstrating a different facet of the CineGrid philosophy of networked extreme media
In Act 1, a sequence of 4K “digital shorts” at 24 frames per second (fps), together with fully mixed synchronized audio, were pulled in real time from network-connected servers in Los Angeles and San Diego.
Thursday, 20 July 2007 Digital Representations of Performing Arts 58
Cinegrid@AES October 25th 2006Cinegrid@AES October 25th 2006
In Act 2, 4K tele-presence was used for interactive video-conferencing and ultra-realistic reproduction of a classical music performance from Tokyo
In Act 2, 4K tele-presence was used for interactive video-conferencing and ultra-realistic reproduction of a classical music performance from Tokyo
Thursday, 20 July 2007 Digital Representations of Performing Arts 59
Cinegrid@AES October 25th 2006Cinegrid@AES October 25th 2006
Acts 3 and 4 were designed to prove the concept:networked, remote audio post-production for digital cinema
by creative teams spread around the world
who demand the highest-quality production values
In Act 34K motion pictures were sent compressed from Tokyo
24-channel non-compressed digital audio was streamed from San Diego
In Act 4The performance system was re-configured
to use uncompressed 2K motion pictures coming from ILM servers in the LDAC facility, synchronized to 24-channel, non-compressed digital audio streaming from San Diego
Acts 3 and 4 were designed to prove the concept:networked, remote audio post-production for digital cinema
by creative teams spread around the world
who demand the highest-quality production values
In Act 34K motion pictures were sent compressed from Tokyo
24-channel non-compressed digital audio was streamed from San Diego
In Act 4The performance system was re-configured
to use uncompressed 2K motion pictures coming from ILM servers in the LDAC facility, synchronized to 24-channel, non-compressed digital audio streaming from San Diego
Thursday, 20 July 2007 Digital Representations of Performing Arts 60
Cinegrid@AES October 25th 2006Cinegrid@AES October 25th 2006
Was a good test of the CineGrid concept of:
Using 4K cameras
Multi-channel, non-compressed audio
Ultra-realistic “live” experiences of music concerts
and …
other kinds of performing arts
To distant audiences in theatres
Connected by high-speed networks
Was a good test of the CineGrid concept of:
Using 4K cameras
Multi-channel, non-compressed audio
Ultra-realistic “live” experiences of music concerts
and …
other kinds of performing arts
To distant audiences in theatres
Connected by high-speed networks
Thursday, 20 July 2007 Digital Representations of Performing Arts 61
Cinegrid@AES October 25th 2006Cinegrid@AES October 25th 2006
"I believe the very high-speed optical networks such as now being deployed by research organizations will become an essential infrastructure for digital cinema production and distribution”
"But, we still have to learn how to integrate systems that creative people can use to make beautiful 4K content - picture and sound - in new ways appropriate for the 21st century. Demonstrations such as CineGrid@AES force us, in a good way, to learn by doing."
"I believe the very high-speed optical networks such as now being deployed by research organizations will become an essential infrastructure for digital cinema production and distribution”
"But, we still have to learn how to integrate systems that creative people can use to make beautiful 4K content - picture and sound - in new ways appropriate for the 21st century. Demonstrations such as CineGrid@AES force us, in a good way, to learn by doing."
Tomonori Aoyama, Professor at Keio/DMC and Chairman of the Digital Cinema Consortium of Japan (DCCJ) said:Tomonori Aoyama, Professor at Keio/DMC and Chairman of the Digital Cinema Consortium of Japan (DCCJ) said:
Thursday, 20 July 2007 Digital Representations of Performing Arts 62
UK National Academic NetworksUK National Academic Networks
UKERNAUK Education and Research Networking Association
Now renamed to JANET
Operate the standard IP serviceWhat most of use use daily
With the rollout of SUPERJANET-5 we now have:
JANET Lightpath Service
Evolved from UKLightSee www.uklight.ac.uk for details
Various research projects
Many coordinated from NeSC and the eSI
UKERNAUK Education and Research Networking Association
Now renamed to JANET
Operate the standard IP serviceWhat most of use use daily
With the rollout of SUPERJANET-5 we now have:
JANET Lightpath Service
Evolved from UKLightSee www.uklight.ac.uk for details
Various research projects
Many coordinated from NeSC and the eSI
Thursday, 20 July 2007 Digital Representations of Performing Arts 63
What is a lightpath?What is a lightpath?
End-to-end Network CapacityPoint-to-point circuits
Capacities from about 50Mbit/s to 10Gbit/s
Presentations
• Ethernet – 100/1G/10G (WAN/LAN)• SDH – STM-1/4/16/64
Implemented over JANET optical transmission infrastructure
including SDH equipment re-deployed from UKLight
End-to-end Network CapacityPoint-to-point circuits
Capacities from about 50Mbit/s to 10Gbit/s
Presentations
• Ethernet – 100/1G/10G (WAN/LAN)• SDH – STM-1/4/16/64
Implemented over JANET optical transmission infrastructure
including SDH equipment re-deployed from UKLight
Thursday, 20 July 2007 Digital Representations of Performing Arts 64
Janet NetworksJanet Networks
JANET Optical Transmisson
JANET IP Research capacityUKLight
AdditionalWavelengths
Thursday, 20 July 2007 Digital Representations of Performing Arts 65
SUPERJANET-5 Integrated with UKLIGHTSUPERJANET-5 Integrated with UKLIGHT
EuropeGEANTUSA
StarLight
YHMAN
C&NLMAN
Clydenet
EaStMAN
NorMAN
NNW
TVN LMN
EMMAN
Bristol
Glasgow
Reading
Warrington
Leeds
LondonT-House
T-City
NIRAN
Dublin (HEANET)
SWERN
LeNSE
Kentish MAN
WREN
EastNet
AbMAN
FaTMANUHI
MidMAN
Thursday, 20 July 2007 Digital Representations of Performing Arts 66
Getting a Lightpath - JANET PolicyGetting a Lightpath - JANET Policy
Subject to capacity and budget availability, lightpaths will be free at point of use up to 1Gbit/s
Tariffs available for 10Gbit/s
Delivered to Institutions at the JANET Access Point
Institutions must commit to extending the lightpath across their networks
See www.ja.net/lightpath for details
Documents shortlyFirst public version in final editorial stage
Service description and application lightpath application form
Subject to capacity and budget availability, lightpaths will be free at point of use up to 1Gbit/s
Tariffs available for 10Gbit/s
Delivered to Institutions at the JANET Access Point
Institutions must commit to extending the lightpath across their networks
See www.ja.net/lightpath for details
Documents shortlyFirst public version in final editorial stage
Service description and application lightpath application form
Thursday, 20 July 2007 Digital Representations of Performing Arts 67
High Performance ComputingHigh Performance Computing
The more power we need the less we can afford it
Not sensible to have massive systems duplicated
Look towards the grid for power
But, what if we need it in real time?
Current technology geared towards batch computing
Some notable exceptionsInteractive visualisation
Computational steering
VLBI
Some CERN LHA experiments
Data reduction in real time
The more power we need the less we can afford it
Not sensible to have massive systems duplicated
Look towards the grid for power
But, what if we need it in real time?
Current technology geared towards batch computing
Some notable exceptionsInteractive visualisation
Computational steering
VLBI
Some CERN LHA experiments
Data reduction in real time
Thursday, 20 July 2007 Digital Representations of Performing Arts 68
Grid Computing in the UKGrid Computing in the UK
National Grid Service (NGS)
NGS-1 4 core sites with compute and TB storage
NGS-2 Officially in pre-production as of 20th June
Sevices …
Job Submission via:Globus, GridSAM, Condor, gLite
or via portals and applications repositories“to simplify job submission for users who do not wish to delve into the technical details of grid computing “
Grid Support Centre to help users
National Grid Service (NGS)
NGS-1 4 core sites with compute and TB storage
NGS-2 Officially in pre-production as of 20th June
Sevices …
Job Submission via:Globus, GridSAM, Condor, gLite
or via portals and applications repositories“to simplify job submission for users who do not wish to delve into the technical details of grid computing “
Grid Support Centre to help users
Thursday, 20 July 2007 Digital Representations of Performing Arts 69
Data Grid in the UKData Grid in the UK
NGS offers:Oracle 10g
Storage Resource Broker (SRB)
OGSA-DAI
VOMS
Oracle 10g – high performance database system
SRB - data storage servicestore your data in geographically distributed locations
access them all using only the logical filename
Reliabilty and efficiency are improved dramatically
OGSA-DAIThe Open Grid Services Architecture Data Access and Integration
middleware that helps users access and integrate data
from separate structured data sources e.g. XML or Oracle
NGS offers:Oracle 10g
Storage Resource Broker (SRB)
OGSA-DAI
VOMS
Oracle 10g – high performance database system
SRB - data storage servicestore your data in geographically distributed locations
access them all using only the logical filename
Reliabilty and efficiency are improved dramatically
OGSA-DAIThe Open Grid Services Architecture Data Access and Integration
middleware that helps users access and integrate data
from separate structured data sources e.g. XML or Oracle
Thursday, 20 July 2007 Digital Representations of Performing Arts 70
VOMSVOMS
Virtual Organisation Membership Service
To help grids manage the authorization of their users
Delegates the approval of users to the to the VO itself
Thus removing the onus upon the end user to register with each resource they might use as part of the VO
NGS resources are in the process of adopting VOMS to authorize NGS members
In the future the number of VOMS-aware resources is likely to increase
Virtual Organisation Membership Service
To help grids manage the authorization of their users
Delegates the approval of users to the to the VO itself
Thus removing the onus upon the end user to register with each resource they might use as part of the VO
NGS resources are in the process of adopting VOMS to authorize NGS members
In the future the number of VOMS-aware resources is likely to increase
Thursday, 20 July 2007 Digital Representations of Performing Arts 71
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Academic Background
Involvement in Performing Arts
My involvement with technology
TechnologyBespoke hardware
Computers
Networks
Software
Engaging with this technologyRight people in the right place
Future
Questions
Thursday, 20 July 2007 Digital Representations of Performing Arts 72
High Performance ComputingHigh Performance Computing
Wavefield Synthesis is a good example
Computationally demanding
Desktop machines and small clusters not capable
Can we have real time grid computing please?
If not, will someone fund a big lorry:With lots of computers
Lots of aircon
Plug in 3-phase power at one end
Gigabit ethernet comes out the other end!
Wavefield Synthesis is a good example
Computationally demanding
Desktop machines and small clusters not capable
Can we have real time grid computing please?
If not, will someone fund a big lorry:With lots of computers
Lots of aircon
Plug in 3-phase power at one end
Gigabit ethernet comes out the other end!
Thursday, 20 July 2007 Digital Representations of Performing Arts 73
Wavefield Synthesis – what, why, when?Wavefield Synthesis – what, why, when?
The limitation of stereo or ambisonic techniquesit only works perfectly well for one listener
positioned on the so-called sweet spot
Thus in common concert environmentsthe intended effect of movement of the sound will in these cases not be heard by a majority of the listeners
Wave field synthesis can overcome this limitation
Can provide a good perceptual localisation in a relatively large listening area
This makes the technique ideal for concert environments
The limitation of stereo or ambisonic techniquesit only works perfectly well for one listener
positioned on the so-called sweet spot
Thus in common concert environmentsthe intended effect of movement of the sound will in these cases not be heard by a majority of the listeners
Wave field synthesis can overcome this limitation
Can provide a good perceptual localisation in a relatively large listening area
This makes the technique ideal for concert environments
Thursday, 20 July 2007 Digital Representations of Performing Arts 74
Wavefield Synthesis – Simplistic overviewWavefield Synthesis – Simplistic overview
Sound is wavesNext wave can be synthesised by a infinite number of small speakersNeed to approximate “infinite”
Sound is wavesNext wave can be synthesised by a infinite number of small speakersNeed to approximate “infinite”
Thursday, 20 July 2007 Digital Representations of Performing Arts 75
Wavefield Synthesis – the realityWavefield Synthesis – the reality
Various experimental speaker arraysVarious experimental speaker arrays
Thursday, 20 July 2007 Digital Representations of Performing Arts 76
Exemplar VenueExemplar Venue
RECOMBINANT MEDIA LABS, SF
Founded to create, research, and portray Spatial Media Synthesis
The science of projecting image and sound objects in 3-dimensional space
RML cultivates radical methodologies for experiential engineering
Exploring performative processes which expand the formal aesthetic and technological boundaries of immersive installation and surround cinema
RECOMBINANT MEDIA LABS, SF
Founded to create, research, and portray Spatial Media Synthesis
The science of projecting image and sound objects in 3-dimensional space
RML cultivates radical methodologies for experiential engineering
Exploring performative processes which expand the formal aesthetic and technological boundaries of immersive installation and surround cinema
Thursday, 20 July 2007 Digital Representations of Performing Arts 77
RMLRML
The San Francisco facility contains:
A flexible black box environment that houses:
A high definition multi-channel audio-visual systemknown as Surround Traffic Control
Full fidelity array consists of a design specification for:10 screens in 360 degrees
supported by an ultra impact 16.8.2 horizontal and vertical sound diffusion system
scalable for all types of rectangular rooms
High Speed Networking
The San Francisco facility contains:
A flexible black box environment that houses:
A high definition multi-channel audio-visual systemknown as Surround Traffic Control
Full fidelity array consists of a design specification for:10 screens in 360 degrees
supported by an ultra impact 16.8.2 horizontal and vertical sound diffusion system
scalable for all types of rectangular rooms
High Speed Networking
Thursday, 20 July 2007 Digital Representations of Performing Arts 78
SummarySummary
Seen the evolution of some technology
The newer technologies require:High performance computing
Very high speed networks
Massive storage
New breeds of creative artists
Many different experts in the technologies
Networking of experts
New theatres
Investment
and much much more
Seen the evolution of some technology
The newer technologies require:High performance computing
Very high speed networks
Massive storage
New breeds of creative artists
Many different experts in the technologies
Networking of experts
New theatres
Investment
and much much more
Thursday, 20 July 2007 Digital Representations of Performing Arts 79
ANY QUESTIONS?
ANY QUESTIONS?