Upload
meryl-golden
View
214
Download
0
Embed Size (px)
Citation preview
SimMillennium Project Overview
David E. Culler
Computer Science Division
U.C. Berkeley
NSF Site Visit
March 2, 1998
March 2, 1998 SimMillennium Overview 2
The Vision
• To work, think, and study in a computationally rich environment with deep information stores and powerful services– test ideas through simulation
– explore and investigate data and information
– share, manipulate, and interact through natural actions
• Organized in a manner consistent with the University setting
March 2, 1998 SimMillennium Overview 3
SimMillennium Project Goals
• Enable major advances in Computational Science and Engineering– Simulation, Modeling, and Information Processing becoming
ubiquitous
• Explore novel design techniques for large, complex systems– Fundamental Computer Science problems ahead are problems
of scale
• Develop fundamentally better ways of assimilating and interacting with large volumes of information– and with each other
• Explore emerging technologies– networking, OS, devices
March 2, 1998 SimMillennium Overview 4
Goals of this talk
• Components of the Project– Community
– Cluster-based Resouces
– Connectivity
– User Interaction
– Computational Economics
• Specific Infrastructure
• Research Agenda
March 2, 1998 SimMillennium Overview 5
Component 0: Community
• An inter-disciplinary community with common interests and shared view of the future– strong momentum in computational science and engineering
=> Session II
– Members of 17 campus units and NERSC in Intel Millennium
– Need and commitment required for participation
– Key subset represented in this proposal
March 2, 1998 SimMillennium Overview 6
Component 1: Resources (Millennium)
• An environment with vast cluster-based computing power and storage (CLUMPS)
behind a personal 3D desktop
NT 3DDesktop
GroupClusterof SMPs
Dept.SMP
CampusCluster
March 2, 1998 SimMillennium Overview 7
Resource Component Support
• Computers via Intel Technology 2000 grant– 200 NT desktops
– 16 department 4-way SMPs
– 8 5x4 Group Clusters,
– 1 ~100x4 Campus Cluster
– PPro => Pentium II => Merced
• Additional storage via IBM SUR grant– 0.5 TB this year => 4 TB
• NT tools via Microsoft grant
• Solaris x86 tools via SMCC grant
• Campus provides Technical staff
• Research provides the prog. and system support
200 Gflop/s
150 GB memory
8 TB disk
March 2, 1998 SimMillennium Overview 8
Key NSF investment: Cluster Network• Transforms large collection of individual resources into a powerful system
– can be focus on a problem
• High Bandwidth– scales with the number of processors (Gb/s per proc)
• Low Latency
• Low Overhead
• Low Cost
• Simple and Flexible
• Almost no errors
• Low Risk
• Today: Myrinet
March 2, 1998 SimMillennium Overview 9
Cluster Research Agenda
• Applications grow into resources– huge range of needs
– require Algorithmic Innovation, Prog. Tools, & Performance
• Dealing Deep Memory Hierarchy – New numerical algorithms on CLUMPs
– New compiler techniques for parallel object language
• Fast Multi-protocol Communication
• Global system at large scale– Unix vs. NT, single system image vs. objects
• Exciting technology turnover– VIA, SANs, Gigabit Ethernet
=> Session III
March 2, 1998 SimMillennium Overview 10
Campus Support Enables Research
• Technical staff develop and deploy common solution and environment– networked systems designer
– unix-based programmer and cluster system admin.
– NT-based programmer and tools
• Technical computing software developed jointly with NERSC
• Participating departments provide system administration and construction costs.
March 2, 1998 SimMillennium Overview 11
Component 2: Connectivity
• Create a richly interconnected pool of resources owned by members of the community– Enable transportation of huge data sets and computation
– Enable remote visualization and collaboration
– Enable extensive sharing of resources
• Expand networking technology
Campus Cluster
CS Cluster
EECluster
CE Cluster
ME Cluster
Astro/Phys Cluster
xport Cluster
BIO Cluster
Econ/Math Cluster
March 2, 1998 SimMillennium Overview 12
NSF Investment: Inter-cluster network
• Gigabit Ethernet connecting group clusters and campus cluster
• Bay Networks provides 70% discount
• Campus provides fiber plant, maintenance, and staff
March 2, 1998 SimMillennium Overview 13
Physical Connectivity
March 2, 1998 SimMillennium Overview 14
Inter-Cluster Research Agenda
• Vastly expands the scope of systems challenge– integrate well-connected resources according application
needs, rather than physical packaging
– resource allocation, management, and administration
• Network bandwidth matches display BW– Protocols and run-time sys. for visualization, media transport,
interaction, and collaboration.
• Community can share non-trivial resources while preserving sense of ownership– Bandwidth translates into efficiency of exchange
– Data can be anywhere
• Important networking technology in its own right.– Layer 3 switching, QoS, VLan
=> Session III, V
March 2, 1998 SimMillennium Overview 15
Component 3: User Interaction
• High-quality 3D graphics emerging on cost-effective platforms– desktops and dedicated cluster nodes
– NERSC team provides modern scientific visualization support
• Gigabit network allows this to be remote.
• New displays create “workbench” environment where large volumes of information can be viewed and manipulated.
• Trackers and Haptic interfaces greatly enhance degrees of user input– 3D capture
March 2, 1998 SimMillennium Overview 16
NSF Investment: UI Technology
• Two Projection Table– large field of view in horizontal (or vertical) orientation
• Phantom Haptic Interface– 3D force feedback
• Motion Tracker– untethered position
• 3D Shutter Glasses– low cost visualization
March 2, 1998 SimMillennium Overview 17
User Interaction Research Agenda
• Expand access to 3D visualization– Explore any data anywhere
– Ease development
• Develop lab-bench metaphor for Viz– two hands, physical icons
• Fast prototyping and exchange through Informal Interfaces– sketching
• Dealing with large volumes of information– lenses, brushing and linking
• 3D collaboration and interaction
=> Session VI
March 2, 1998 SimMillennium Overview 18
Component 4: Computational Economy
• How is this vast, integrated pool of resources managed?
• Traditional system approach: empower global OS to provide “optimal” allocation to blind applications– predefined metric, tuned to fixed workload
– ignores the inherent adaptation of demand
• Computer Center– charge => director-to-user feedback according to cost
• Economic view: decentralized allocation according to perceived value– pricing => user-to-user feedback
– compatible niches,sense of control, cooperation
– idea has been around, why now?
March 2, 1998 SimMillennium Overview 19
Research Agenda
• Natural fit to academic structure– members want control over own resources, and each has
varying needs that far exceed dedicated resources
– incentive for maintaining resources up to par
• Address partial or delayed information, component failure, and user satisfaction from the start
• Framework for elevating design from resources to services
• Rich body of theory, little empirical validation– experts in several parts of the community
• New paradigm for algorithms & perf. Analysis
• Complex, large-scale systems
March 2, 1998 SimMillennium Overview 20
Basic Approach• Desktop an active agent conducting automated negotiation for resources• Servers provide resources to highest bidders
– monitor usage and enforce limits within remote execution environment– placement based on economic advantage
• Higher level system functions are self-supporting– resource availability, brokering, directories
• Useful applications packaged as services– may charge more than resources cost
March 2, 1998 SimMillennium Overview 21
NSF Investment: Staff Support
• Provide enabling technology and let it evolve– monitoring, enforcement
– exchange
– negotiation tools
• Integrate it into users enviroment
• Tools and measurements to determine effectiveness
March 2, 1998 SimMillennium Overview 22
Integrated Research Agenda• Advance the State of Computational Science and
Engineering– immerse a community in a computationally rich environment with
the right tools: algorithms, programming & system support
– Path to exploiting novel techniques and technology
• Explore design techniques for robust large-scale distributed systems– economic (or ecologic) approach
• Explore new ways of interacting with information– large paste-ups, two hands, sketching, 3D collaboration
• Investigate new technology– SMP nodes, gigabit Ethernet, SANs, VIA
– NT, dCOM, Java beans, directory services
– workbench displays, 3D icons, haptics, position sensors
March 2, 1998 SimMillennium Overview 23
Perspective
• Highly leveraged investment in a large scale infrastructure for studying problems of scale
• Deep commitment across the campus
• Sense of ownership and participation
• Rich research agendaOverall 11M$ Budget
Breakdownin K$
6,000
1,850
1525
1000
400
300
600
Intel
NSF
UCB
IBM
Microsoft
Sun
Synoptics