Upload
juana
View
33
Download
0
Embed Size (px)
DESCRIPTION
Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds. Rhys Davies, IT Director University of Leeds & Chairman YHMAN Ltd Friday 8 th November 2013 Location: Aldwark Manor Golf & Spa Hotel, Alne Nr York Meeting: 22nd Annual NYHDIF Conference. - PowerPoint PPT Presentation
Citation preview
November 2013 slide 1
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
Big Data & Collaboration
The Theory, Practice & Opportunity, a view from the University of Leeds
Rhys Davies, IT Director University of Leeds & Chairman YHMAN Ltd
Friday 8th November 2013
Location: Aldwark Manor Golf & Spa Hotel, Alne Nr York
Meeting: 22nd Annual NYHDIF Conference
November 2013 slide 2
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
Agenda
• Introduction & background
• The Theory
– Approach, Vision & Plan
• The Practice
– Preparation, Timing, Execution & Outcomes
• The Opportunity
– What this might mean for Health Sciences
– Where we are now
• What next ?
• Questions ?
November 2013 slide 3
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Theory …
My Role
Approach
Vision
November 2013 slide 4
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Theory …
My Role
Approach
Vision
“To create a Collaborative Centre of Excellence
which can be used for Research Computing
for the benefit of Academia, Health, Commerce and the greater good;
to be built on a mutually beneficial ‘model’
with the belief that by sharing assets (equipment, intellect, funding)
we can deliver more, better, cheaper for all concerned”
November 2013 slide 5
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Preparation
What were the required ingredients ?
People:
Shared Vision; Skills; Energy; Appetite for Risk; Trust
Process:
Collaboration; Consistency; Secure; Sustainability
Technology:
Network; Compute; Storage; Data; Service
November 2013 slide 6
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Preparation, Timing & Execution
What were the required ingredients ?
People:
Shared Vision; Skills; Energy; Appetite for Risk; Trust
Process:
Collaboration; Consistency; Secure; Sustainability
Technology:
Network; Compute; Storage; Data; Service
November 2013 slide 7
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Networks #1
Beyond the UK JANET Services offer onward connect to:
UK Internet Peering
Europe (GEANT Network)
US Internet, Abilene & ESnet2
Japan (NI) & China (CERNET)
Beyond the UK JANET Services offer onward connect to:
UK Internet Peering
Europe (GEANT Network)
US Internet, Abilene & ESnet2
Japan (NI) & China (CERNET)
The National and International picture
Secure,Free for research
November 2013 slide 8
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Networks #2
The Local picture
•Secure,•Resilient,•‘Limitless’ bandwidth, •Free at point of consumption
November 2013 slide 9
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #1
• The UK’s first triangulated datacentre
• One of the worlds’ largest spanning datacentres
• Has the unique capability to span to more than 3 hubs
November 2013 slide 10
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #2
• Innovative
• Award winning
• Secure
• Resilient
• Virtual
• Highly available
• Linked to HPC
November 2013 slide 11
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #2
• Innovative
• Award winning
• Secure
• Resilient
• Virtual
• Highly available
• Linked to HPC
• This also covers Security; Storage; Services
November 2013 slide 12
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, next step…
Fundamental Building Blocks are in place…
The physical network
The shared virtual data centre = ‘safe haven’
•The Supercomputer
•The skills necessary to exploit
November 2013 slide 13
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #1
November 2013 slide 14
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #2
November 2013 slide 15
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #3
N8 HPC ‘Approach & Way of Working’
Research Impact & Industrial Growth
Research-led (Vertical) Themes Network
Research-led (Vertical) Themes Network
Cross-cutting (horizontal) themes in methods and techniques
Cross-cutting (horizontal) themes in methods and techniques
• Institutional, specialist research computing support
• Specialist Facility Support• N8 Industry Innovation Forum• Business Engagement Teams• Research Computing Training• Doctoral Training (CDT)
• Institutional, specialist research computing support
• Specialist Facility Support• N8 Industry Innovation Forum• Business Engagement Teams• Research Computing Training• Doctoral Training (CDT)
Centr
e o
f Exce
llence
Infr
ast
ruct
ur
eR
ese
arc
h
November 2013 slide 16
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #4
N8 HPC • EPSRC funded in March 2012
– Capital– First year set-up and running costs
• Aims– Establish a Tier 2 HPC facility– Develop a computational science research network– Share support and training expertise– Develop collaborative links with Tier 1 partners– One stop shop for business – key themes for
engagement
• Future running costs underwritten by partners
November 2013 slide 17
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #5
5312 2.6Ghz Intel Sandy Bridge cores2:1 blocking QDR infiniband4GB/core (256 cores @16GB/core)174TB Lustre parallel filesystemCentOS/Redhat 6.3 based.SGE scheduler, Intel/GNU Compilers, OpenMPI/IntelMPI/MVAPICH2Locally- and centrally-provided software.
Co-located with 4500-core Leeds HPC
Purchased through Esteem framework agreement: SGI hardware, Alces integration
#291 in June 2012 Top500
November 2013 slide 18
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #1, Proctor & Gamble
November 2013 slide 19
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #2, Proctor & Gamble
November 2013 slide 20
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #3, BBC
Opportunity & Challenge•Relocation of BBC to Salford
– Exploring opportunities to deepen relationship with University of Manchester
•“Making Musical Moods Metadata”
– 128,000 audio files
– 53 transformations (classifying mood as f(time))
– On current tech: Over a year of processing time
Outcome
•Over a year of processing down to 12 hours.
•“The entire dataset was processed in only 12 hours, creating the world's largest time-varying musical feature database. Their combination of cutting-edge facilities and outstanding support was of huge benefit in getting the project completed and we look forward to working with them again.” – Chris Baume
November 2013 slide 21
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #4, A VRE
November 2013 slide 22
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #5, Secure Storage ITC
HE Community Storage •Secure•UK Located•Cost effective•Functionally richer than anything else in the current market•Institutional scale•Single point of access to all data sources•Authenticated to your systems
“UNIVAULT”
November 2013 slide 23
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Opportunity
November 2013 slide 24
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
University of Leeds
Leeds Teaching Hospitals Trust
SharedInfrastructure
InvestmentClinical
InfrastructureInvestment
Research Infrastructure
Investment
PPMN8 / HPC
Consent systemData extraction
The Opportunity, What might this mean for Health #1
November 2013 slide 25
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
University of Leeds
Leeds Teaching Hospitals Trust
SharedInfrastructure
InvestmentClinical
InfrastructureInvestment
Research Infrastructure
Investment
PPMN8 / HPC
Consent systemData extraction
PhenobankingBiobanking & Analysis
The Opportunity, What might this mean for Health #1
November 2013 slide 26
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Opportunity, What might this mean for Health #2
November 2013 slide 27
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Opportunity, What might this mean for Health #3
Figure 3
November 2013 slide 28
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
What Next ?
Shared Virtual Data Centre– we have our first customer and are investigating options
with other HE
N8 High Performance Datacentre- we are looking to develop the next iteration of this
Secure Storage in the cloud- we are starting to market this and looking for BETA testers
Big Data Collaboration with LTHT- MRC decision expected this month
November 2013 slide 29
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
Questions…
Thank you
???