Upload
edith-mclaughlin
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
Electronic Visualization Laboratory, University of Illinois at Chicago
Pacific Research Platform Scalable Visualization and Virtual-Reality Team
www.evl.uic.edu
Maxine D. Brown Electronic Visualization Laboratory (EVL)
University of Illinois at Chicago
Electronic Visualization Laboratory, University of Illinois at Chicago
EVL’s Visualization and Virtual Reality Collaboration Hardware and Software Help Teams Manage “Big Data”
CAVE 1992
SAGE (2004-2014) and SAGE2 (2014-present)
TacTile 2008
CAVE2 2012
“Star Wars” 1977
Electronic Visualization Laboratory, University of Illinois at Chicago
EVL Networking: External PRP Connectivity
• UIC connected to StarLight at 100Gbps
• EVL connected to UIC backbone at 100Gbps– 2x40Gbps to the
CAVE2/Omegalib (36Mpixel stereo / 72Mpixel mono)
– 10Gbps to Cyber-Commons/SAGE2 (18Mpixel)
Electronic Visualization Laboratory, University of Illinois at Chicago
Current Tools to Retrieve and Move DataNot User Friendly!
• Interactive “human in the loop”: Using URL with HTTP/HTTPS protocols– Can be slow – Web not designed to transfer terabytes– If a browser is needed, it can be an issue, since storage machines are often not
interactive – need to download on laptop/workstation, then transfer to the storage device (might not be possible if very large)
• Non-interactive / Command line– scp/sftp
• Ubiquitous on Unix systems• Tuned for security• Slow – not tuned for performance
– curl/wget• Ubiquitous• Not secure• Not fast either (web server)
• Current network speeds to our offices– ~1Gbps within US, if sites are well tuned and have decent storage– 10-20 Mbps otherwise
Electronic Visualization Laboratory, University of Illinois at Chicago
EVL Use Case #1: SAGE2 Scalable Amplified Group Environment
Over 40+ Sites Worldwide
• Middleware to access, display, and share high-resolution digital media on scalable resolution display environments
• Based on web technologies
• Multi-touch interaction (one or many people)
• Push laptop screens or windows onto a wallsage2.sagecommons.org
Electronic Visualization Laboratory, University of Illinois at Chicago
SAGE2 Data Sizes and Frequency of Transfer
• Data Types– Documents, pictures, movies (accessed as URLs)– Video streams
• Data Sizes– Variable sized datasets from scientists and artists
• Frequency of Transfer– Ad hoc
Electronic Visualization Laboratory, University of Illinois at Chicago
SAGE2 Collaborators Exchanging Data
• Collaborations with academic, government lab, non-profit (museums) and industry research institutions – regional, national and international
• Local and remote digital media sent as URLs from the SAGE2 web server
http://sage2.sagecommons.org/community-2/
Electronic Visualization Laboratory, University of Illinois at Chicago
SAGE2 Speeds Achieved To Date
• SAGE2 tested to leverage high-speed networks1 display node 2 display nodes 4 display nodes 6 display nodes
Bandwidth from a NodeJS server
1.8 Gbps 4.3 Gbps 7.0 Gbps 9.3 Gbps
A single SAGE2 Javascript server can send at least 10Gbps with enough clients (display nodes).
Electronic Visualization Laboratory, University of Illinois at Chicago
EVL Use Case #2: OmegalibHybrid Visualization Environment
https://github.com/uic-evl/Omegalib
EVL CAVE2Calit2-QI StarCAVE
• Currently virtual-reality middleware• To be extended as a scalable, modular, platform/device-independent
framework for scientific visualization, to span 2D personal devices to large 3D immersive displays, cluster-based low-latency streaming, and local to remote cloud computing. Luxor data provided
by Calit2-QI, UCSD
Electronic Visualization Laboratory, University of Illinois at Chicago
EVL Use Case #3: Large File Transfer
• Retrieved ~200GB application from a CAVE2 in Australia to a CAVE2 in Chicago– Australian system secured behind 2 gateways (two logins required to access the storage)– No fast transfer tools installed
• UIC Routing– Initially went through the campus production network, but updated to research network– Research network not configured for jumbo-frame end-to-end (the end-points were, not the
core)– Issues with routing, DNS, ...
• Ended up using ‘scp’ from command line through campus network and Internet2 WAN• Speed: ~20Mbps, 24 hours to transfer data – Should be 30 min at 1Gbps, 3 min at 10Gbps
Monash University CAVE2 showing “Fifty Sisters”
Electronic Visualization Laboratory, University of Illinois at Chicago
EVL Use Case #4: Classic SAGE + UltraGridUltraGrid software enables low-latency, high-quality video network transmissions
• GLIF 2014 Telemedicine demo: Sending video streams among UIC, UCSD, and REANNZ at the New Zealand workshop site.
• 100Gb network from Seattle to NZ was so new, there were issues with jumbo-frame and reliability
• Error correction adds some bandwidth (FEC)• People want multiple streams to enable awareness across sites (wide-angle
front, wide-angle, back, speaker closeup, …)• Streams first sent to UCSD, then retransmitted using an UltraGrid reflector.
Pushed ~9.5Gbps among the three sites (compressed 4K and uncompressed HD streams)
www.ultragrid.cz/en
UIC UCSD Calit2-QI REANNZ
Electronic Visualization Laboratory, University of Illinois at Chicago
EVL Use Case #5: SENSEI 360° Stereo Video CameraThe Sensor Environment Imaging (SENSEI)
Instrument is a real-time, image-acquisition, sensor-based camera system to capture spherical,
omnidirectional, stereo 3D video and still images of real-world scenes, to view in networked,
collaboration-enabled, virtual-reality systems
When built, SENSEI will have ~100 sensors of 2-8 Mpixels each, and, in motion-capture mode of 30fps, will generate 360-1440
Gigapixels/minute. A pixel is 3-4 bytes, resulting in TB/min
Stereo still-photo panoram of Luxor taken with CAVEcam (precursor to SENSEI)
www.evl.uic.edu/sensei
Electronic Visualization Laboratory, University of Illinois at Chicago
Network Issuesa.k.a. What’s Screwed Up?
• Storage devices not often accessible on the network edge – hidden behind firewalls, multiple gateways or login system
• Storage not on the right network• Network misconfigurations: no jumbo-frame
network end-to-end• UDP enables low latency but can be problematic for
networks• Firewalls not configured for UDP
Electronic Visualization Laboratory, University of Illinois at Chicago
In an Ideal World, We’d Have…
• Intelligent, scientist-friendly tools to access “Big Data” files from home, office, lab (The goals of SAGE2 and Omegalib)
• Fast protocols and transfer tools– Widely deployed (no sys admin required)– Handle multiple types of streams: video, data, audio,
tracking,…
Electronic Visualization Laboratory, University of Illinois at Chicago
In an Ideal World, We’d Have…
10 Gbps “BIG
” SCIENCE
10 Mbps O
FFICE
TIME TOMOVE A DVD5 Seconds 1 Hour
TIME TOMOVE A TB12-16
Minutes10 Days
Gigabyte
1 Billion
Megabyte
1 Million
Terabyte1
Trillion
Petabyte1
Quadrillion
100 Gbps “BIG
” SCIENCE
1.6 Minutes
Electronic Visualization Laboratory, University of Illinois at Chicago
• Funding from Federal agencies, industry and non-profit institutions
• Fostering early adoption by supporting user communities
• Providing educational experiences to students, who receive jobs upon graduation
EVL: Thank You!