Commercialisation at NeSCAn operational perspective

Dr Rob BaxterSoftware Development Group

Manager

NeSC Review, 30/09/2003

Overview

NeSC project strategyThe GCPs and ediktSome highlights of the yearHow do we do it?Where are we going?

NeSC project strategy

The Grid must be built on standardscf. the Internet, the World Wide Web

Core middleware is fundamentalcf. TCP sockets libraries, http servers

Our strategy has beenidentify e-Science application needsdevelop standards and core middlewarebuild applications, leveraging middleware

This has proved very successful for e-Science

and provides a solid foundation for e-Business

Software projects at NeSC

NeSC has two flavours of projectdevelopment - the GCP projectsresearch and development - the edikt programme

GCP projectsfunded by £3.6m EPSRC/DTI (+ equal in-kind)allow focused development on key e-Science problemsdemand direct engagement with business

ediktfunded by £2.3m SHEFC RDGallows exploration of new ideas, some good, some bad

NeSC project portfolio

GCP:SunDCG - m/wOGSA-DAI - m/wGridWeaver - fabricMS.NETGrid - m/wFirstDIG - appPGPGrid - appBRIDGES - appODD-Genes - demo

edikt:• BinX - m/w• Eldas - m/w• AstroBinX - app• Osage - app

[chronological starts]

Collaborators roll-call

In the last two years we’ve worked with

IBM, Sun Microsystems, HPOracle, MicrosoftFirst plc, Pepper’s Ghost Productions ltd, Raytheon Company inc

andAstroGrid, myGridESNW, NEReSC, LeSC, GSCScottish Centre for Genomic Technology & Informatics, MRC Human Genetics Unit

Some highlights

OGSA-DAI£1.8m, NeSC, EPCC, ESNW, NEReSC, IBM, Oraclecore data access middleware based on GGF standards

OGSA-DAI team helping write the specifications

in use in e-Science projects AstroGrid, myGrid, FirstDIG, BioSimGrid, BioGrid (Jp)

1000 downloads reached last weekfollow-on project planned (DAIT, £1.5m)gained EPCC/NeSC entry to the Globus Alliance

only UK site alongside Argonne, ISI and PDC

Some highlights cont.

MS.NETGriddemonstration and training of OGSI on .NETfull house at AHM 2003!

great interest at the NeSC booth too

delivered course & demos 9-10/09/03 @ eSIinvited by MS to lecture on Grid/.NET, Budapest, Oct

FirstDIGEPCC, FirstBus South Yorkshireapplication of OGSA-DAI and related data mining

to draw together real, disparate, dirty data sources to add real value

“the results of this exercise will revolutionise the way we do things in the bus industry”

Darren Unwin, Divisional Computing Manager, FirstBus South Yorkshire

Some highlights cont.

ODD-GenesEPCC, GTI, MRC HGUa demonstration of OGSA-DAI and SunDCG in actionthe highlight of the NeSC booth at AHM 2003!

“This project has demonstrated how Grid technologies can be used to enable true e-Science - discoveries that would not otherwise have been achieved without this infrastructure in place”

Professor Peter Ghazal, Director, GTI

How do we do it?

The NeSC/EPCC SDG thinks like a companywe do fixed-priced contracts to deliver software X against customer requirementsmust plan and execute properly or we get cost overrunbut we also operate on the technology bleeding edge…

…so we multiply all our task estimates by 2.3

Software Development Group: 30 staffSDG Manager3 Project Managers, 2 Architects5 Principal Consultants/Team Leaders19 software developers

Well-defined project lifecycleselection, planning, execution, delivery and review

Project selection

Two flavours, two methodsGCP Projects

Commercial Group prospect based around CommStratproposals developed with clients and SDG project staff“project licence” agrees handover from CG to SDG

ediktedikt management define proposals based on work with our e-Science stakeholdersproject proposals reviewed by Advisory Board

Project planning

Older projects used standard development planNewer projects use more comprehensive project management plan

a quality plan template based on NASA SEL and other sources (CMU SEI, Microsoft, Oracle)covers all aspects of the project

development plan risk and issue monitoring project tracking metrics documentation and coding standards QC and QA

Project planning cont.

Leading edge development needs flexibilityWe use phased development models

staged delivery good where scope is ill-defined and we expect requirements

to change/evolve, e.g. OGSA-DAI, SunDCG define scope in quarterly blocks (say)

design to schedule good where scope is well-understood and initial requirements

are well defined, e.g. MS.NETGrid, FirstDIG work against prioritised requirements until effort runs out

evolutionary prototyping good for demos or exploring requirements, e.g. ODD-Genes

Project execution

All project teams haveproject leaderdeveloperstechnical reviewers

Bigger projects will have an architect…e.g. OGSA-DAI, edikt projects

…and/or a Review Boarde.g. OGSA-DAI, SunDCG, edikt

Weekly team meetings are expectedaction lists, task breakdowns, risk & issue lists

Project execution cont.

Based on standard document templatesrequirements, component design, test…

Use standard toolse.g. Rational Rose, TogetherSoft, JUnit, ant

Test early, test oftenaim for high unit test coveragehave developed automated test setups for distributed middleware over course of GCP projects

Monitor overruns, apply triagereview plans regularly, revise if required

Delivery and review

All projects postmortemedwhat was good? what was bad? what can we learn?

For customer-led projects, delivery is easy

site visit, installation, traininglimited period of free supportlonger term support contracts always offered

Middleware is a bit differentmonitor web downloadsset up web-based bug reportingdisseminate, disseminate, disseminate!

Where is this taking us?

Solutions for e-Science today……are solutions for e-Business tomorrow

universal data access and integrationuniversal data format descriptionWeb and Grid services

WSDL, SOAP, HTTP, Java, .NET

distributed resource managementdistributed, heterogeneous systems integration

Our projects reflect our vision for the commercial Grid

In summary

NeSC’s software development programme has been - and continues to be - a great successWe leverage off existing strengths at EPCC

the institute that thinks it’s a company

Professional planning and execution is keybacked by an unrivalled CS research base

International recognitionfrom a standing start to the Globus Alliance in 2 yearsSunDCG software already distributed by Sun

Real business impact“this will revolutionise the way we do things”

Darren Unwin, FirstBus South Yorkshire