Upload
jorge-sweeney
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
ANET Workshop
History of Internet Audio Experiments at McGill (and elsewhere)
Jeremy R. CooperstockCentre for Intelligent Machines
Centre for Interdisciplinary Research in Music Media and Technology
McGill University
ANET Workshop
Some Early (Pre-)History
Eve Schooler: Distributed Music: A Foray into Net Performance (Sept. 1993)
synchronized three real-time streams from different hosts; delays in the order of 200 ms difficult for performers to be listeners
Paul Hoffert: CyberSoiree (Feb. 1996) ATM-based technology for audio and video streaming of a four-way jazz performance delay >0.5s delay but musicians earned to compensate through extensive practice
Dimitri Konstantas: Distributed Musical Rehearsal Studio (May 1996) ATM based distributed rehearsal with conductor at different location from musicians 80 ms one-way delay for audio-video synch; echo resulted in "extreme confusion"
Seiji Ozawa: Opening Ceremony Nagano Winter Olympics (1998) conduct choruses on 5 continents: singers in Sydney, New York, Beijing, Berlin, False Bay time lag adjustor used to eliminate satellite delay
ANET Workshop
World's first Internet AC-3 streamMontreal-New York for the AES 107th Convention, Sept. 26, 1999
ANET Workshop
System Diagram
1.5
Mbps
1.5 Mbps
SGI Indy
PC ClientCisco IP/TVPC Server
SGI IndyNetwork
Dolby encoder
AC-3 (640 Kbps) (5.1 channels,16 bits @ 48kHz)
coded in AES/EBU
McGill UniversityMontreal, Canada
NYU Cantor HallNew York, USA
ANET Workshop
In action
Receiver
Transmit Receive,check, playback
Send AudioQueue
Sender
Receive AudioQueue
Read
ANET Workshop
Recording Studio that Spanned a ContinentMontreal-Los Angeles for the AES 109th Convention, Sept. 23, 2000
McGill Redpath Hall, Montreal
USC Norris Theatre, LA
12 channels, 96kHz, 24
bits/channel
ANET Workshop
Some Modern History
Internet2 Initiative: World's First Remote Barbershop Quartet (Nov. 2000) multi-location quartet; each of the 4 singers in different cities, conductor in 5th network delay variances prevented singers from hearing each other or conductor
Internet2 Initiative: Music Video Recording via Internet2 (Nov. 2000) multi-location music video recording session using real-time streaming video musicians simultaneously connected via timing tracks to a mixing board
Chris Chafe: QoS Enabled Audio Teleportation (Nov. 2000) CD quality sound (750 kbps) of 2 separated musicians in Dallas streamed to Stanford musicians played "together" in same space (Stanford) but delay was severe
John Wawrzynek: Network Musical Performance (May 2001) gestural coding (e.g. MIDI) used to manage data for distributed musical performance musicians at Berkeley and CalTech, playing on MIDI keyboards; local feedback only
ANET Workshop
"Frères Jacques" experimentMontreal, RISQ 2000 Conference
"low-latency" MPEG-2 videoconferencing
ANET Workshop
"Playing Together" experimentNew York-Ottawa, Remote Masters class, Dec. 8, 2000
used Litton MPEG-2 codec @ 10 Mbps
Pinchas Zuckerman, Columbia U, NYC
Wu Ji, Canarie ARDNOC, Ottawa
ANET Workshop
Quality vs. Latency
near-equivalent quality of encoded video at greatly reduced bandwidth
but... compression takes time
0
50
100
150
200
250
300
uncompressedSDI
M-JPEG/DV MPEG-4
Bandwidth (Mbps)
ANET Workshop
Tolerated Synchronicity
compress transmit
100 20 4030 50 7060 80ms
speed of light: Mtl-LA
net latency Mtl-LA
networks
smallensemble
large ensemble
lip synch
musicalactivity
* This figure is a theoretical “best-case” scenario. Actual performance may vary based on local hardware. The manufacturer makes no warranties, either express or implied, and will not be held liable for damage resulting from use of the product. Certain restrictions may apply. Offer void where prohibited by law. Batteries not included.
MPEG*acquire
ANET Workshop
Interactive Success!
Chris Chafe: SoundWIRE (August 2001) streaming pro audio from remote locations for musical collaboration UDP/IP streaming: good results between Armonk, NY and Stanford round trip delay 125ms, musicians able to "catch-up" during pauses
ANET Workshop
For those with DSL at home...For those with DSL at home...
yeah, right!(tu n'es pas
sérieux!)
ANET Workshop
So what does it take?
much bigger network pipes ("abusive bandwidth") need Fast Ether (100Mbps) for audio GigE (1000Mbps) for video more efficient network topologies – hops add delay
ANET Workshop
So what does it take?
much bigger network pipes ("abusive bandwidth") need Fast Ether (100Mbps) for audio GigE (1000Mbps) for video more efficient network topologies – hops add delay
more powerful hardware CPU speed (fast process switching, especially for mixed media) pro-audio interfaces with small buffers
ANET Workshop
So what does it take?
much bigger network pipes ("abusive bandwidth") need Fast Ether (100Mbps) for audio GigE (1000Mbps) for video more efficient network topologies – hops add delay
more powerful hardware CPU speed (fast process switching, especially for mixed media) pro-audio interfaces with small buffers
more masochistic researchers
ANET Workshop
Distributed Violin DuetMcGill-Université de Montréal for RISQnet Conference, Nov. 21, 2001
connected two Montreal universities via IP over 1 Gbps fiber link
uncompressed audio and video
< 30ms latency
ANET Workshop
"The medium is the message" Marshall McLuhan
Cooperstock's Corollary:
"The medium must be faster than the message."
ANET Workshop
Size does matterMontreal-Ottawa (NRC), Distance Teaching Trial, Feb. 20, 2002
connected McGill to National Research Council (Ottawa) with SDI video and multichannel 96kHz/24bit audio
display on 50" plasma screen (near life-size)
ANET Workshop
Distributed Jazz JamMontreal-Stanford (CCRMA), Research Demo, Oct. 24, 2002
uncompressed audio and video @ < 50ms latency
ANET Workshop
Remote Video Interpretation (Sign Language)Montreal-Fredericton (UNB), research trials, 2003
DV (patient, physician) exchanged with remote sign-language interpreter
physician's setup
patient's setup
ANET Workshop
Milestones Ahead
multicast traffic shaping region-of-interest client selection multichannel echo-supression/cancellation
Self-Signal Isolation: Echo-suppression
the big problem in videoconferencing same issue for video and vibrosensory data
achieving high-fidelity distributed interaction, at levels of presence and latency that support the
most demanding applications and doing so in spite of sensor and bandwidth limitations