V R

E

UK e-Research, Services and Portals(integrating components and

interfaces)

Rob Allan CCLRC e-Science Centre,,Daresbury Laboratory,

Warrington WA4 4AD, UK

(with thanks to Rob Crouchley and Chuck Severance)

VRE

V R

ESummary

Virtual Research Environment projects were funded by HEFCE/ JCSR in 2004. I will report on progress in a few of these developing and delivering services to diverse research communities. e-Research has many special requirements, such as using the National Grid Service resources (funded by JCSR in 2003), but many service are being found to overlap with digital information and e-Learning. This permits the exploration of common interfaces, in particular through portal frameworks such as Sakai. Joint work involving UK and US developers will be described.

V R

EThe UK National Grid Service (JCSR+EPSRC)

V R

ENGS Core (JCSR)

2x Compute Clustersgrid-compute.leeds.ac.ukgrid-compute.ox.ac.uk

•64 dual CPU Intel 3.06 GHz (1MB cache) nodes•2GB memory per node •2x 120GB IDE disks (1 boot, 1 data) •Gigabit network •Myrinet M3F-PCIXD-2 •Front end (as node) •Disk server (as node) with 2x Infortrend 2.1TB U16U SCSI Arrays (UltraStar 146Z10 disks)

•PGI compilers •Intel Compilers, MKL •PBSPro •TotalView Debugger •RedHat ES 3.0

2x Data Clustersgrid-data.man.ac.ukgrid-data.rl.ac.uk

•20 dual CPU Intel 3.06 GHz nodes •4GB memory per node •2x120GB IDE disks (1 boot, 1 data) •Gigabit network •Myrinet M3F-PCIXD-2 •Front end (as node) •18TB Fibre SAN ( Infortrend F16F 4.1TB Fibre Arrays (UltraStar 146Z10 disks)

•PGI compilers •Intel Compilers, MKL •PBSPro •TotalView Debugger •Oracle 9i RAC •Oracle Application server •RedHat ES 3.0

V R

EHPCx Supercomputer Facility

V R

ENGS Applications

V R

EDeployment and Uptake Challenges

To Service Providers:• Defining the Service – Computational and Data Services• Which Middleware? - Globus, OGSA, WSRF, EGEE/LCG?• Integrating Helpdesk Support – UK + Europe (EGEE)• Adding new sites -Service Level Descriptions,

monitoring• Licensing – applications, particularly commercial ones• Security, firewalls etc.

To Users:• Training and Awareness – what can it do for me?• Education - Getting people to use the “Grid” • Making it easier to use – better/easier client access for

non-technical users• Creating new applications and expressing requirements

– Grid “stretch”

V R

EThe Grid “Client Problem”

Grid Core

Consumer clients: PC, TV, video, AG

Workplace: desktop clients

Portable clients: phones, laptop, pda, data entry…

Middleware

e.g. Globus

Grid Core

Many clients want to access a few Grid-enabled resources

V R

EInstitutions need Autonomy and Security

Host – client relationship

Example solution suggested by

Web server - browser

Communication must be initiated by client because of firewall around client’s institution. Can use a proxy or gateway.

V R

EUsability and Human Factors

• Growing recognition of the need to design a behaviourally appropriate interface to the Grid;

• Rick Stevens’ Access Grid and work on human factors issues;

• Lot of industrial knowledge here, ergonomics etc. needs to be built on;

• Trust, ethics, security etc. – may make it impossible to re-use and analyse data

• Usability Task Force is taking a lead (white paper and CfP - EPSRC);

• Job of scientists already hard – need tools that do not make it harder!

V R

E

Digital libraries & documents

groups-to-information

groups-to-facilities

people-to-people

Communication,Collaboration

Services

Distributed,media-richinformationtechnology

Remote instruments

http://www.scienceofcollaboratories.org/

Science of Collaboratories

NSF Funded ITR

V R

ESo a Virtual Research Environment?

Requirements:

Easy to install

Familiar interface

Personalisable

Work through firewalls

Extensible functionality

Persistent

Pervasive

Usable

Secure

V R

EWhy have a VRE ?

“to make the use of e-Science technologies, methodologies and resources easier and more transparent for users than simply developing bespoke applications on a generic infrastructure toolkit (such as Globus GT2 or OGSI/WSRF).” We need to:

• Bridge the gap between different types of technology (database management, computational methods, sensor Grids, networks, Condor resources, visualisation systems, collaborative working, Access Grid, etc.);

• Provide an environment to enhance the programmability and usability of such a Grid by integrating work from a number of ongoing research projects;

• Add value to the Grid by implementing a VRE on the JCSR clusters and resources at other e-Science Centres.

V R

ESome basic VRE Functions

VRE must take care of many things behind the scenes:

• Authentication and authorisation (Shibboleth and Permis in line with JISC proposals…);

• Shared development of content by staff using content management and editing tools:– Access to middleware resources and

documentation,– Access to training materials and resources,– Access to support, consultancy and other services

• Access to Grid Services - user access via pre-defined tools and applications to the UK e-Science Grid;

• Data access – e.g. using Storage Resource Broker;• Access to broadcasts – e.g. on the Access Grid

network;• Management functions - for experts to maintain the

system and deploy new applications.

V R

E

Middleware/Software Library

Access GRID

Security Authorisation Authentication

Text Mining/ Data services

UK GRID Services

D

JJISC PortalJISC Portal

Portal Management

Semantic GRID Services

VLE Portal VRE

Portal

Awareness Raising Resources

Workshops

Possible functionality/ content of a VRE

V R

ECommon Service Component Framework

V R

E

Traditionally, there are some differences•Locus of control •Existing versus emerging information•Fixed versus fluid agenda•Different tools in use

Similarity and Differences

VRE Phys

ics

VRE ChemistryIE Social

ScienceTeaching Learning

VisualizationGrid Computing

Annotation

QTIScormAttendance

ChatDiscussionResources

Repository

Shared Data

V R

EPersonalising the Environment

Choose from the set of service components available:

•Move from a provider or institution focused set of capabilities to an environment where users “assemble” their environment to suit their needs.• System that maps to how I think and operate so that things are made easier for me. How do I bend this tool to suit the way that I work? Especially as my skills as a user improve.• Move bits around arrange the way you like. This is both things like my own folder arrangements and things like accessibility (i.e. how I want to “see” these things)

V R

EChallenge I: Integrating the People!

• If people from different areas do not work together they develop their own independent solutions to what has been recognised as common problems.

• It is not enough to show the service component diagrams. People need to cross the traditional “pillars” to share ideas about how services are defined, deployed and used.

• Open Service Framework for Research and Education – working group debating the way forward.

• VRE Programme is deploying shared tools:– Sakai portal “worksite” -

http://collab.sakaiproject.org under “eResearch”– Wiki – http://www.grids.ac.uk/eResearch – JISCMail – eResearch@jiscmail.ac.uk

V R

EDanger of diverging “personal” Environments

CollaborativeVLE/VRE/IE

Requirements

VLE “Experts”

VLE Customers

VRE “Exp

erts

”

VRE Cust

omer

s

IE “Experts”

IE Customers

When we treat Virtual Learning, Virtual Research, and Information Environments as different, we end up developing divergent environments which satisfy similar requirements in very different ways based on the experts who are funded to produce the VRE, VLE, or IE solutions.

Each expert group is often influenced by a different field of research: VLE’s are influenced by Educational Technology experts, VRE’s are often influenced by Computer Scientists, while IE’s are influenced by Library Sciences.

V R

EView from the Users’ Perspective

CollaborativeVLE/VRE

Requirements

VLE “Experts”

Customers

VRE “Exp

erts

”

Custom

ers

IE “Experts”

Customers

As painful as it may be, the VRE, VLE, and IE experts must begin to coordinate so that some point in the future, users don’t have environments with completely different approaches to the same problem.

V R

EThe VRE needs to be more than a Web

page

Why should it be different?

• Like the Web, persistent and pervasive, but:• It provides a managed environment, giving secure

access to autonomous Grid services, providing resources, based on user requirements;

• It uses diagnostic/ background data to orchestrate the material for each individual (via session management/ profiling services);

• It will be specific to the needs of groups of scientists (virtual organisations), providing new routes to e-Science;

• The technology will be easily extendable to include all new tools – and link to commercial software;

• It could be an early adopter of new WSRF/ GT4 and portlet standards.

V R

E

Someday the VRE will not be just available from a browser

a. pure html web page

b. web page based, but with browser enhancements

c. browser extension

d. dedicated desktop network client

e. extensible desktop application platform

f. common desktop application

Better user experienceIncreased productivity

More complex to buildDifficult to keep up with changing technology

V R

EOptions

• Provide heavyweight functionality (Globus?), but only on Grid-enabled hosts;

• Implied need for client-server software architecture, e.g.:– Web-based portal with familiar browser– Client programming library, API in C, C++ Java,

Perl, Python, R etc. – Ability to link to existing applications/ GUIs – e.g.

MATLAB– Command-based shell interface– Drag and Drop interface (a la Mac)

• Need a published set of services on Grid hosts – OGSA model, registry, semantics;

• Need easy development and deployment framework for applications and client tools, e.g. using Web services - encourage community contribution via an open process.

V R

EChallenge II: Grid-enable Services

Need to Grid-enable data and other services, e.g. those supported by JISC:

• RDN, Resource Discovery Network: http://www.rdn.ac.uk • See also Subject Portals Project:

http://www.portal.ac.uk/spp • MIMAS, Manchester Information and Associated Serices:

http://www.mimas.ac.uk • JORUM, JISC Online Repository for Learning and

Teaching Materials: http://www.jorum.ac.uk • EDINA: http://www.edina.ac.uk • AHDS, Arts and Humanities Data Service:

http://www.ahds.ac.uk • UK Data Archive: ESDS the Economic and Social Data

Service http://www.data-archive.ac.uk • UKOLN: http://www.ukoln.ac.uk

V R

ESome UK Research Resources

Existing resources need to be service enabled:

• Access Grid Nodes (e-Science Centres);• Course Content (University and Training Institutions);• Condor pools of workstations (University and Teaching institutions);• Resource Discovery Network resources (JCIE)• AHDS (AHRB) and e-SS (ESRC) and related training and awareness material, e.g. REDRESS;• Directories: Z-Directory (UKOLN), Z39-50 target directory (Index Data), RSS-express

(UKOLN), OAI Data providers (OAI), IESR (JISC)• Text mining service (BBSRC), Data Curation Centre and any other specific research

resources funded in partnership with Research Councils;• Resources referenced in the JISC subject resources guides• Subject gateways; Data services; Learning and teaching;• Support services.• Tools referenced in JISC Collections publications list• National Grid Service nodes (JCSR). Supercomputing facilities such as HPCx, CSAR

(managed by EPSRC):• Data Archive and MIMAS (ESRC);• Protein Data Bank (Hosted by Wellcome Foundation at EBI);• Large-scale facilities such as SRS, ISIS, Diamond (hosted at CCLRC) and associated

scientific data collections;• LHC Data Grid (PPARC);• NERC Data Centres and CEH;• Telescopes, e.g. via eSTAR services (PPARC);• CURL libraries, British Library, National Museums, etc.• Others such as British Geological Survey, UK Met. Office, Hadley Centre.

V R

EExample I: Sakai VRE Demonstrator

CCLRC Daresbury Laboratory with Universities of Lancaster, Oxford and Portsmouth

• A range of tools for collaboration, data access and Grid computing

• Available in a Web-based portal• Using the Sakai portal framework to integrate with

Virtual Learning and group collaboration tools• But: focused on standards

– JSR-168 Java portlet standard and Web Services for Remote Portlets, WSRP

V R

EWe are currently exploring the “best of breed” 2nd generation

portal –Sakai http://www.sakaiproject.org • Originated from U. Michigan (the CHEF project) with partners at

U. Indiana, MIT, Stanford.• Mellon Foundation Funding with OKI standard interfaces• Rich set of Collaboration and Educational tools with templates

for tool creation• We have proposed using this as a VRE “front end” by extending

the framework to a Service Oriented Architecture• Evaluation exercise funded by JISC

http://www.grids.ac.uk/Sakai/sakai_doc.pdf• Using in the ReDRESS project http://redress.lancs.ac.uk We will be:• Using technology: JSR-168 to create plug-in portlets• Using technology: WSRP to link to remote portlets and content• Using Grid and Web services for SOA including UDDI registry

Portal Technology

V R

Ehttp://www.sakaiproject.org

V R

EExtending the Portal Framework

We proposed to extend the Java portlet-based frameworks in several areas:

• Collaborative Research (Lancaster)– identity and group management– XML schema for collaborative session

• Security (Oxford)– address extensible authentication and authorisation– to embrace Shibboleth and PERMIS technology– already looked at CAS, VOMS, GSI

• Distributed services in SOA (Daresbury)– using Web/ Grid service technology– JSF interface generator

• Linking to Peer-to-Peer technology (Portsmouth)

V R

EExample II: GROWL VRE Toolkit

CCLRC Daresbury Laboratory with Universities of Cambridge and Lancaster

• A C/ C++ library toolkit to link to “heritage” scientific applications and GUIs;

• Use Grid resources and data from your apps!• Lightweight to install – uses Web service client;• No firewall problems, but functionality may be limited;• Access same underlying services as in portals, so

enriches the user interface;• Could be replaced with a “drag-and-drop” interface,

e.g. using QT on Linux.

V R

E

• GROWL: Grid Resources on Workstation Library;• Lightweight Web services based toolkit;• Provides and interface (wrappers) for C and R

applications.

R wrapperC client

C service

Application

SOAP over HTTP

Using the Grid from within a standard Application

V R

EStandard Protocols, Interfaces and Tools

Essential to adopt agreed and de facto standards:

• Standards-based portals: JSR-168, etc.– API standardisation but adding “integration API”;

• Basic look and feel standardisation and accessibility: CSS;

• Federating/ aggregating portals and tools - WSRP: user control over assembly of many sources;

• Ability to write portable full-featured tools

– Java• e.g. Swing

– Eclipse– Flash MX

SOAP, WSDL, UDDI

PortletPortlet

Portlet

Portlet

Service ServiceService

Service

Service

JSR-168, WSRP

UI to “heritage” application via command line or GUI and programming library.

V R

E

Me – Rob Allan: r.j.allan@dl.ac.uk

VRE Programme Manager – Maia Dimitrova: m.dimitrova@jisc.ac.uk

GOSC Director - Neil Geddes: n.i.geddes@rl.ac.uk

NGS Grid Coordinator - Andrew Richards: a.j.richards@rl.ac.uk

VRE Programme -http://www.jisc.ac.uk/index.cfm?name=programme_vre

Other Web sites -

http://www.rcuk.ac.uk/escience

http://www.grid-support.ac.uk

http://www.ngs.ac.uk

http://www.grids.ac.uk/ETF

Helpdesk – E-mail: support@grid-support.ac.uk

Contacts