Introduction to the Minnesota Supercomputing …...Storage • High Performance Storage – 4.1 PB...

© 2015 Regents of the University of Minnesota. All rights reserved.

Introduction to theMinnesota Supercomputing Institute

for Advanced Computational Research

Jim Wilgenbusch & Nancy Rowemsi.umn.edu → About → Staff

© 2015 Regents of the University of Minnesota. All rights reserved.

Overview• Why: Mission• Who: Organization • Where: Locations• What: Resources and Services• How: Eligibility, Access, and Accounts

– Getting Started on MSI Systems (Hands On)– Brief Intro to Unix and Useful Commands (Hands On)

© 2015 Regents of the University of Minnesota. All rights reserved.

Organization• MSI is an academic unit of the University of

Minnesota under the Office of the Vice President of Research

• About 40 full time employees and work study students support MSI

• MSI serves more than 600 PI groups with over 3,000 users

• Our current Director is Claudia Neuhauser

© 2015 Regents of the University of Minnesota. All rights reserved.

Organization

Office of the Vice President for Research

Advanced Systems

Operations

- Common Services- HPC Systems

- Storage Systems- Hosted Services

Scientific Computing Solutions

-- Optimization -- Benchmarking-- HPC Research

Workflow & pipeline Development

Application Development

Solutions

- Custom App Dev

- System Programming

Research Informatics Solutions

-Informatics education

-Informatics research

-Informatics services

-Life Science Computing

User Gateway Group

-- User Support Lead

-- User Training-- On Boarding

-Communications-- Outreach

© 2015 Regents of the University of Minnesota. All rights reserved.

VisionFoster innovation and discovery

through advanced research computing

© 2015 Regents of the University of Minnesota. All rights reserved.

© 2015 Regents of the University of Minnesota. All rights reserved.

© 2015 Regents of the University of Minnesota. All rights reserved.

LabsUMN-TC East Bank, Walter Library• Scientific Development and Visualization Laboratory (SDVL)

– Workstations– Tutorials/Workshops

• LCSE-MSI Visualization Laboratory (LMVL)– Virtual Reality– Large Screen– Remote Visualization– Escorted Access

© 2015 Regents of the University of Minnesota. All rights reserved.

LabsUMN-TC East Bank, Nils Hasselmo Hall• Basic Sciences Computing Laboratory (BSCL)

– Stereoscopic Monitors– Stereoscopic Projection System

UMN-TC East Bank, 717 Delaware• Biomedical Modeling, Simulation, and Design Laboratory

(BMSDL)– Stereoscopic Monitors

UMN-TC St. Paul, Cargill Building• Computational Genetics Laboratory (CGL)

– Tutorials/Workshops

© 2015 Regents of the University of Minnesota. All rights reserved.

Locations• Main Offices – Walter Library, 5th Floor• HPC and infrastructure systems are housed in the basement of

Walter Library• MSI also has five labs:

– Scientific Development and Visualization Laboratory (SDVL)– LCSE-MSI Visualization Laboratory (LMVL)– Basic Sciences Computing Laboratory (BSCL)– Biomedical Modeling, Simulation, and Design Laboratory

(BMSDL)– Computational Genetics Laboratory (CGL)

© 2015 Regents of the University of Minnesota. All rights reserved.

Services

Consulting

Batch HPC

InteractiveHPC

Web Portals &Databases

Data Storage

© 2015 Regents of the University of Minnesota. All rights reserved.

HPC Systems

Mesabi: (recent)Cores: 16,848 Intel HaswellMemory: 67 TBAccelerators: 80 K40 Nvidia gpGPUsPeak: 675 TFlops

Itasca: Hewlett-Packard 3000BLCores: 8728 Intel Nehalem CoresMemory: 31 TB of memoryPeak: 100 TFlops

HPC systems require Service Units

© 2015 Regents of the University of Minnesota. All rights reserved.

Mesabi

© 2015 Regents of the University of Minnesota. All rights reserved.

Batch HPC Demo

© 2015 Regents of the University of Minnesota. All rights reserved.

Interactive HPC

• Mesabi– Cores: 16,848– Memory: 67 TB– Accelerators: 80 K40

gpGPUs– Peak: 675 TF

• Itasca– Cores: 8,744– Memory: 31.3 TB– Peak: 100 TF

Environments • iSub:• NICE:• Citrix for Windows• Jupyter Notebooks

© 2015 Regents of the University of Minnesota. All rights reserved.

Interactive HPC -- Demo

© 2015 Regents of the University of Minnesota. All rights reserved.

Storage• High Performance Storage

– 4.1 PB– 48GB/s read/write– Available on HPC resources

• Hadoop Cluster– 40 compute nodes– 1.2 PB Hadoop HDFS

• Tier-2 Storage– 3.1 PB– Available via Amazon’s S3 interface– Available anywhere in the world

• Archive Storage– 3.5 PB tape-based storage – offline storage

Supercomputing Institutefor Advanced Computational Research

© 2009 Regents of the University of Minnesota. All rights reserved.

Library● DRUM, Data

Repository for the U of M

Storage Options at UMN

You● laptop● Mobile

Department● Workstation● Departmental

Servers

MSI● Panasas● Tier-2 CEPH

OIT● Google Drive● Isilon ● Block Storage

Supercomputing Institutefor Advanced Computational Research

© 2009 Regents of the University of Minnesota. All rights reserved.

Storage Options at UMNPurpose Google

DriveOIT

IsilonOIT

BlockMSI

PanasasMSI

Tier-2Dept

StorageLaptop/ Desktop

Big Data ✅? ✅ ✅ ?

High Performance ✅ ?

Share access ✅ ✅ ?

Archival (very long-term) storage ✅ ✅? ?

Access on Campus Laptop/Desktop ✅ ✅? ?

Access on anywhere Laptop/Desktop/Mobile

✅ ✅? ✅? ?

Access on Servers ✅ ✅ ✅? ✅? ?

Legally protected data (Coming) ⌧ ⌧ ⌧ ⌧ ⌧ ⌧ ?

Supercomputing Institutefor Advanced Computational Research

© 2009 Regents of the University of Minnesota. All rights reserved.

GlobusWhat is it good for?• Move data between sites across WAN• Web GUI driven• Move LARGE directory trees with drag and drop• Runs in background

How to use• Get Gobus account:• Register your certificate ID with Globus endpoints• Use web GUI to drag and drop between endpoints

www.globus.org

Supercomputing Institutefor Advanced Computational Research

© 2009 Regents of the University of Minnesota. All rights reserved.

Globus Home Page: start hereglobus.org

120 PB and

counting!

Select:Sign Up

Supercomputing Institutefor Advanced Computational Research

© 2009 Regents of the University of Minnesota. All rights reserved.

© 2015 Regents of the University of Minnesota. All rights reserved.

Storage Demo

© 2015 Regents of the University of Minnesota. All rights reserved.

Portals & DatabasesMultiple “omics” platform

Emphasis on Mass Spectrometry based Proteomics

© 2015 Regents of the University of Minnesota. All rights reserved.

Web Portal -- Demo

© 2015 Regents of the University of Minnesota. All rights reserved.

Portals & DatabasesMinnesota Biodiversity Atlas

Working with the Bell Museum and Zooniverse to organize and annotate images of specimens.

© 2015 Regents of the University of Minnesota. All rights reserved.

Portals & DatabasesFood Protection and Defense Institute

• FIDES – ID potential

disruptions to food supply

– ID food system disruptions using open news media

• FIDES extension to water supply data

© 2015 Regents of the University of Minnesota. All rights reserved.

Software• Approximately 400 software applications

– msi.umn.edu > Resources > Software• Classified into three service levels

– Primary, Ancillary, Minimal• msi.umn.edu > Resources > Software > Software

Service Levels• Users may install software locally• We can purchase software with justification

© 2015 Regents of the University of Minnesota. All rights reserved.

Software Restrictions• Access Restrictions

– User list > Contact Help Desk• Time Restrictions

– Limited Licenses– Calendaring System

• Check documentation for individual software requirements to see if this applies.

• Software on limited systems

© 2015 Regents of the University of Minnesota. All rights reserved.

User Services• Help Desk• Consulting• Tutorials and Workshops• Classes• Code Development• Porting• Parallelization• Visualization

© 2015 Regents of the University of Minnesota. All rights reserved.

Consulting Services

• MSI has 24 consulting staff• 17 consultants have an advanced degree• We provide 10-20 workshops and tutorials annually• 1-2 hour meetings on computational or informatics methodologies• Informational sessions for proposal development

© 2015 Regents of the University of Minnesota. All rights reserved.

Collaborations• Extended involvement with an analyst funded by a grant

mechanism• 46 projects are supported by MSI annually• Collaborations involve staff who have knowledge of:

– Specific research domains– Current analytical methods– Statistics– Computing– Application Development

© 2015 Regents of the University of Minnesota. All rights reserved.

Service Costs• Free for UMN faculty and their associates• Free for other MN academic institutions• Fee for hosting UMN faculty systems• Fee for software development• Fee for service for external organizations

© 2015 Regents of the University of Minnesota. All rights reserved.

Account Structure• Principle Investigator (PI): Controls everything in the

group• Users: Added by the PI• Group Administrators: Users within a group that can do

everything the PI can do except add other Group Administrators

© 2015 Regents of the University of Minnesota. All rights reserved.

PI Eligibility• Faculty members at the University of Minnesota• University of Minnesota academic professionals with

department approval• Faculty researchers at other accredited institutions of

post-secondary education in the state of Minnesota

© 2015 Regents of the University of Minnesota. All rights reserved.

Account Duration• MSI accounts are renewed annually

– Renewal period opens the first week of October– Resource allocations must be renewed each year

• Accounts and allocations are good through the calendar year• Additional resources may be requested during the year

– HPC Allocations Committee meets four times a year– Below threshold resource allocations are handled through the

MSI Help Desk

© 2015 Regents of the University of Minnesota. All rights reserved.

Access• System access information is available at msi.umn.edu >

Access > Sign Up• For current users, access through msi.umn.edu > MyMSI• For building access, visit the MSI reception desk• Information can be found in the General Information page for

each lab via msi.umn.edu > Resources > Labs• For help, email help@msi.umn.edu or call x6-0802

© 2015 Regents of the University of Minnesota. All rights reserved.

Service Units Allocations

• Three Allocation Requests• Group A: 70K - All accounts

– Fairshare set to 70K• Group B: 70K to 280K - MSI Review

– Fairshare set to 280K • Group C: >280K - Committee Review,

Annually– Fairshare set to approved allocation

© 2015 Regents of the University of Minnesota. All rights reserved.

Data Storage Allocations• Three Groups• Group A (Default 150 GB)

– No review• Group B (<5 TB)

– Uses that much now OR– Provide specific justification– Reviewed by MSI

• Group C (>5 TB)– Provide specific justification AND– Estimate how much is needed AND– Estimate how much is actively used AND– Estimate how long data will be stored at MSI– Reviewed by MSI

© 2015 Regents of the University of Minnesota. All rights reserved.

MSI Web Pages• MSI Home Page

msi.umn.edu• Software

msi.umn.edu/sw• Password Reset

msi.umn.edu/password• Tutorials

msi.umn.edu/tutorial• FAQ

msi.umn.edu > Resources > FAQ

© 2015 Regents of the University of Minnesota. All rights reserved.

Questions?• MSI Help Desk is staff Monday-Friday from 9:

00 AM – 4:00 PM

• Walk-in help is available in Walter 587

• Call x6-0802 (1-612-626-0802)

• Email help@msi.umn.edu