UCT eResearch - Presentation for IT reps

University of Cape Town

Overview for IT Managers – 12 May 2015UCT eResearch

eResearch UCT

Anthony Beitz

Director UCT eResearch

www.eresearch.uct.ac.za

12 May 2015

21st century research discovery through advanced computing and ICT



Agenda

1. Introduction to eResearch2. eResearch @UCT3. Examples of eResearch at UCT4. Accessible capabilities & infrastructure



Emergence of Research Methodologies



What is eResearch?

eResearch is 21st century research discovery through the application of advanced computing and IT



It is transformational research:

Second image courtesy of Dr. Govinda Poudel, with permission.Third image courtesy of Electronic Visualization Laboratory & Dept. of Psychiatry, UIC, with permission. Photo by Philip Chan, Monash Uni.



It is accelerated research:

Brains of ~100 study subjects (controls, pre-symptomatic and Huntington’s Disease sufferers) visualised at once!

Data: N. Georgiou-Karistianis (Monash Uni.) Visualisation: Y. Benovitski (Monash Uni.)



MRI Scanner

Collect

GPU Cluster

Compute

CAVE2TM

Comprehend

eResearch version of a microscope

Dat

a Pl

atfor

m –

Col

lect



Research Data Commons

Communicate

Virtual Lab

Collaborate

eResearch version of a microscope

Compute

Collect

Comprehend

Dat

a Pl

atfor

m –

Col

lect

Dat

a Pl

atfor

m –

Col

lect

Dat

a Pl

atfor

m –

Col

lect



Research Data Deluge

Data generation rates:

• Next Generation Sequencer > 200 TB / year

• Direct electron detectors > 350 TB / year (equivalent to synchrotron beamlines)

• Large Hadron Collider 15 PB / year

• Square Kilometer Array 0.3-1.5 EB / year

Laboratory based

Large facilities



Research Acceleration & TransformationIMPLICATIONS

Need to grow access to infrastructure, capability and tools

Pace and scale increasing

• Technology is accelerating the pace and scale of research

• Large-scale data requires more structured approach data management approach

KEY CHANGES

Expert guidance and training to leverage new approaches

New tools and methodologies

• New and more powerful instruments

• Digital recognition of text, speech and imagery

• Crowdsourcing and citizen science

Swift capability to establish and support collaborations

Internationalisation

• Research funding is increasingly based on strong international collaborations

• Internationalisation is introducing new complexities in health, ecology and economy (epidemics, climate change, Global Fin. Crisis)Slide courtesy of Nathan Bailey & Lisa March (iSimplify)



“Scholarly practices across an astounding wide range of disciplines have become profoundly and irrevocably changed by the application of advanced information technology. This collection of new and emergent scholarly practices was first widely recognized in the science and engineering disciplines….humanities continue to showcase some of the most creative and transformative examples of the use of information technology to create new scholarship.”



http://www.canberra.edu.au/research/ucresearch/e-research



Transforming research practice

• Unsuccessful structure solution of PlyC structure

• However, raw data archived in MyTardis, the Monash bioscience research data repository

• New technological advancements in 2010 and easy access to the archival data led to the PlyC structure being solved

Monash’s Dr. Sheena McGowan (under the direction of Prof. James Whisstock and Assoc. Prof. Ashley Buckle)

PlyC

http://www.quantumdiaries.org/



About Researchers• Research is about “doing things we don’t know how to do”

They work in an interpretive mode– Characteristics include: an iterative process; open-ended; and

thrive on ambiguity – Service providers try to remove ambiguity = breakdown in

communication and understanding

• Their requirements and goals will change over time, and very likely won’t be well known at the start

• Generally more loyal to their research community than their institution

• May require an ICT capability for only a short period of time• Very resourceful and driven by research outcomes



Creative Solutions• Solutions are not always

“designed/engineered” – they are often “grown” via an iterative process

• If researchers engage all the way along the iterative process -- they will very likely engage at the end

• The research challenges are unique and therefore the solutions are often “unique” - no one size fits all

Slide courtesy of Prof. Paul Bonnington (Monash Uni)



Engaging with Researchers• It’s basically about the relationship and demonstrating that

you are listening and delivering ongoing value• Never push solutions or technologies• Partner with Researchers and go on the journey of the

unknown together• Solutions should be led and owned by the Researcher

– Agile methodologies and mindsets lend themselves to this outcome

• Work in with their community• Reduce the barriers• The three T’s of an innovation environment: Talent,

Technology and Tolerance (for ambiguity and risk)



eResearch @ UCT



History of UCT eResearchBegan with UCT’s ICT Services’ (ICTS) Executive Director, Sakkie Janse van Rensburg, searching for a better way to support researchers2009: UCT ICTS staff trained in HPC2010: Built 1st cluster and visited Australia to investigate eResearch2011: HPC-trained staff focused on researchers, UCT researcher appointed to work with ICTS2013: Best-of-breed eResearch model identified, global search2014/Q1: Core partnerships and eResearch working group formed2014/Q2: Business case developed, buy-in by UCT’s Senior Leadership Group attained, initial eResearch team formed2014/Q3: Strategic plan developed2014/Q4: Interim director appointed2015/Q1: Governance established



eResearch Responsibilities

LegendA – AccountableC - ConsultedI – Informed



eResearch Centre

Research Office• Advocacy• Collaboration

ICTS• Data Storage• Compute• Cloud• Networks

DVC-Research• Leadership• Governance• Strategy• Resourcing

The Library• Research visibility &

discovery• Research data

management

Faculties• Various capabilities

Researchers

Core capability partnerships

Capability delivery



UCT eResearch partners with research groups to accelerate and transform research, connecting them to the most appropriate hardware, software and services to sustain that capability

Collect Compute Comprehend

Collaborate Communicate Customise

Creating a world-class environment for 21st century discovery



As a partner and broker UCT eResearch will connect researchers with world-class expertise and facilities (local or external) in:

• Collection and management of research data

• Modelling, simulation and data processing through HPC

• Comprehension of big data through visualisation and data science techniques

• Promotion of collaborative research through virtual labs and cloud resources

• Dissemination of research outcomes (including data and workflows)

• Development of customised research software, hardware and services



Resource Limits• Requirements for advanced ICT

research infrastructure are huge and growing

• Advice and fundamental infrastructure are freely available

• For needs beyond fundamental infrastructure, UCT researchers engage early with the eResearch Centre to estimate required funding and to incorporate it into a grant

Imag: permabit.com/data-affordability-gap/



UCT eResearch does not want to try to be the ultimate Swiss army knife for researchers



Target Structure for UCT’s eResearch Centre

eResearch Centre Steering

Committee

Directorate

Researcher Liaisons

Research Infrastructure

Projects

Fledgling Research

Technology Platforms

Virtual Labs & Collaborative Environments

Human Capital Development

DVC-R Exec. Dir. Research Office

NB: The lines are not indicative of line reporting

Exec. Dir. ICTS University Librarian



Functions• Directorate

– Strategy, leadership, facilitate organisational change, comms and admin

• Researcher Liaisons– Act as brokers between researchers and capability providers

• Research infrastructure projects– Plan & execute research infrastructure projects

• Fledgling Research Technology Platforms– Provide skilled resources, specialised infrastructure and relevant training

• Virtual research labs & collaborative environments– Provide interim support for virtual research labs and collaborative envs

• Human Capital Development– Co-ordinate staff and researcher training in new research cyberinfrastructure



Examples of eResearch at UCT



Examples of eResearch at UCT



SKA - answering some of science’s biggest questions

• One of the largest scientific projects ever undertaken to answer questions such as:

• What is dark energy?

• Was Einstein right about gravity?

• Is there life elsewhere in the universe?

• eResearch is supporting storage, compute, development of new visualisation tool

• Exascale arriving soon…

www.skatelescope.org



Enabling South African CERN researchers to partner with their international colleagues

• CERN made up of a massive globally dispersed community of researchers

• Successful international research collaborations are dependent on a good video conferencing experience

• South African researchers had a poor experience

• UCT eResearch connected CERN with national service providers to bring video conferencing up to an good level



• Key research projects e.g. SKA and space geodesy

• Use of HPC facilities have tripled in the past year

• 300 researchers across campus assisted by HPC

• 33 citations (and counting)

Image: Stephen Williams Photography

High Performance Computing



Facility Support: Olympus Virtual Microscope

The old: slide boxes

The new: Olympus VS120 Microscope

eResearch:

• Worked with the UCT’s Clinical Lab Services department to set up the servers and the database

• Configured the system for various access scenarios including student, departmental and public access



Facility Support: Centre for Imaging and AnalysisProvides the instrumental resources and expertise for structural analysis in all disciplines.

acquisition processing interpretation dissemination

Workflow:



Facility Support:

• Nikon fluorescent microscope for live cell imaging

• eResearch supported storage, remote data analysis and collaboration

Clinical Research Centre Molecular and Cell Biology Tissue Culture Facility

• Clinical, laboratory and data management systems needed to pass an FDA or EMA audit.

• Software and computational requirements met by eResearch



Human Heredity and Health in Africa

• First Pan African Bioinformatics Network in Africa: H3ABioNet & H3Africa Consortium

• UCT eResearch Centre, in collaboration with Information and Communication Technology Services (ICTS) and the Computational Biology Group, will be developing a data storage archive solution

• The data will include genomic data and access will be controlled according to internationally accepted standards.



Accessible capabilities & infrastructure



Current capabilities & infrastructure

• Training– HPC (beginner & advanced levels)– Software carpentry (Python,R, Git, ..)

• Research data storage• High Performance Computing• Virtual machines and databases• Project Managers, BAs, Devs, … for research projects

(through cost recovery)• Assistance in research grant proposals (research

infrastructure)



IT Infrastructure in Support of Research

Ashley [email protected]

Saturday 15 April 2023



UCT Core Network & Data Centres



UCT Bandwidth

• 1.2 Gbps International on WACS• 10 Gbps Local via SANREN



UCT High Performance Computing (HPC)

1. Hex Cluster Nodes• Dell C6145 x 9 = 18 nodes• 4 sockets with 16 cores each• 1152 cores• 2.2TB RAM

2. HPC Cluster Nodes• HPBL460c x 19 = 19 nodes• 8 – 12 cores each• 176 cores• 648 GB RAM

Mellanox Infiniband 6036 SX Interconnect

Dell C6145 compute Node

HP BL460C compute Nodes



UCT High Performance Computing (HPC)

3. GPU Nodes• SuperMicro M2090 x 3 = 3 nodes• 4 sockets with 16 cores each• Nvidia TeslaM2090 & K40M (9856 GPU

cores) • 36 CPU cores• 192 GB RAM

4. High Memory Nodes• Dell R820 x 2 = 2 nodes• 8 – 12 cores each• 176 cores• 648 GB RAM

Dell C6145 & R820 compute Nodes



UCT HPC Storage

1. /home• 2 TB Storage (NFS)

2. /scratch (old)• 21 TB Storage (NFS)

3. /scratch (high speed)• 92 TB (BGFS)

BackBlaze Storage Pod

Dell MD3060 Storage unit



UCT Research Data Storage



UCT Research Data Storage Benefits:

• Highly discounted for research – R3450 per TB or R34 500 per 10 TB• Once-off payment• Data is backup & replicated to UFS & UWC data centres• Access your data from anywhere in the world• Collect & store data from your instruments directly to the central research data store.• Contact: - [email protected]



Coming capabilities and infrastructure

• New visualisation tool & platform• Federated access• New data centre• New compute cluster – 1000+ cores• Training in data science, research data

management (in league with the Library)• African Research Cloud (ARC)



…in summary



Monash eResearch Centre• eResearch is 21st century research discovery through the application of advanced

computing and IT

• UCT eResearch partners with research groups to accelerate and transform research, connecting them to the most appropriate hardware, software and services to sustain that capability

UCT eResearchAdvanced computing and IT for accelerating and transforming research

Collect Compute Comprehend

Collaborate Communicate Customise



See: http://www.eresearch.ac.za

eResearch AfricaUniversity of Western Cape

9-13 November 2015

http://www.eresearch.ac.za/





The Team Anthony Beitz (Director)

Timothy Carr

Jenny Dixon (Admin)

Piet Helm

Heine de Jager

Andrew Lewis

Chris Mtshengu

Adrianna Pinska

Jason van Rooyen

Ashley Rustin

Natalie Simon



UCT eResearch

Email: [email protected]: www.eresearch.uct.ac.zaPhone: +27 (76) 595 9616

http://www.eresearch.uct.ac.za/

Education

UCT eResearch - Presentation for IT reps