Estimatingthevalueandimpactof

NectarVirtualLaboratories

ReporttoNectar

DrKimSweeny

DrMashaFridman

ProfessorBruceRasmussen

VictoriaInstituteofStrategicEconomicStudies

VictoriaUniversity

September2017

1

Contents

1.Introduction 2

2.TheNectarproject 3

3.Methodology 7

4.TheVirtualLaboratories 13

4.1BiodiversityandClimateChangeVirtualLaboratory 13

4.2.CharacterisationVirtualLaboratory 15

4.3GenomicsVirtualLaboratory 16

4.4.HumanitiesNetworkedInfrastructure 18

5.DataonusageofNectarVirtualLaboratories 23

6.EstimatingthevalueandimpactofNectarVirtualLaboratories 26

7.FurtherassessmentofthevalueandimpactoftheCharacterisationVirtualLaboratory 35

8.QualitativevaluationofNectarVirtualLaboratories 39

9.Summaryandconclusions 47

References 49

Appendices 53

1. VirtualLaboratoriesfundedbyNectar 532. CVLimagingfacilitypartners 54

2

1.Introduction

In2016theVictoriaInstituteofStrategicEconomicStudies(VISES)atVictoriaUniversitywascommissionedbytheNationaleResearchCollaborationToolsandResourcesproject(Nectar)toestimatethevalueandimpactofNectarVirtualLaboratories(VLs).

NectarprovidesanonlineinfrastructurethatsupportsresearcherstoconnectwithcolleaguesinAustraliaandaroundtheworld,allowingthemtocollaborateandshareideasandresearchoutcomes,whichwillultimatelycontributetoourcollectiveknowledgeandmakeasignificantimpactonoursociety.

Itprovidessupportforanetworkof14VLswhicharedomain-orientedonlineenvironmentsthatdrawtogetherresearchdata,models,analysistoolsandworkflowstosupportcollaborativeresearchacrossinstitutionalanddisciplineboundaries.

GiventheresourcesavailableandthecentralroleofVLsintheNectarproject,itwasagreedthatthemostappropriateapproachtoestimatingthevalueandimpactofNectarwastoconcentrateonestimatingthevalueandimpactof4VLs,namely

• BiodiversityandClimateChangeVirtualLaboratory(BCCVL)• CharacterisationVirtualLaboratory(CVL)• GenomicsVirtualLaboratory(GVL),and• HumanitiesNetworkedInfrastructure(HuNI)

MostVLsfundedbyNectarhaveonlybeenactiveforafewyearsandarestillintheirgrowthstages.AnevaluationoftheiroverallimpactandvaluemightbestbedonefromtheperspectiveofsomeyearsinthefuturewhentheVLsareinamorematuregrowthphase.Thereforetheanalysisandconclusionsdrawninthisstudyshouldbetreatedaspreliminaryanddependsignificantlyontheassumptionsmadeaboutfuturegrowthpaths.

ItbecameclearduringthecourseofthisstudythatwhileVLssharemanyfeaturesincommon,theydiffersignificantlyfromeachotherintermsoftheservicestheyprovidedtotheirtargetcommunities.Furthermoretheydifferinhoweasyitistoexpresstheirvalueandimpactintraditionaleconomicterms.Consequently,wehaveadoptedanumberofquantitativeandqualitativeapproachestorevealingvalueandimpactandhaveusedtheCharacterisationVirtualLaboratoryasacasestudytoanalysethisinmoredepth.OurapproachtoHumanitiesNetworkedInfrastructure(HuNI)hasbeenexploratoryinnaturewiththeaimofidentifyingdifferentapproachesthatmightbemorefullydevelopedinafuturestudy.

3

2.TheNectarproject

NectarhasitsoriginsinCommonwealthgovernmentinitiativestosupportnationalresearchinfrastructure.TheNationalCollaborativeResearchInfrastructureStrategy(NCRIS)wasestablishedbytheCommonwealthGovernmentin2004andcurrentlyprovidesfundingfor27activeprojectsacross222institutionsemployingover1700technicalexperts,researchersandfacilitymanagers.NCRISfacilitiesareusedbyover35,000researchers,bothdomesticallyandinternationallyandalsoattractfundingfromorganisationsinthehighereducation,government,non-profitandindustrysectors(DepartmentofEducation2017).

SinceitsestablishmenttherehavebeenanumberofreviewsandroadmapsoftheNCRISprogram.

TheStrategicRoadmapforAustralianResearchInfrastructure,releasedinAugust2008,updatedasimilarroadmapin2006butplacedincreasedemphasisoneResearch

“inrecognitionofthepervasiveandunderpinningrelevanceofICTtoresearch.Ascollaborativeresearchincreases,eResearchisprovidingthemostinfluentialandeffectivewayofenablinginstitutionstoworktogether,usingsharedinfrastructure,resourcesandpolicies.”1

Inits2009/10BudgettheAustralianGovernmentannouncedaSuperScienceInitiativetofurthersupportresearchinfrastructurebyaddressingtheprioritiesidentifiedinthe2008StrategicRoadmap.TheSuperScienceInitiativewasfinancedthroughtheEducationInvestmentFund.Nectarwasestablishedin2009bytheAustralianGovernmentfollowinga2009/10Budgetannouncementof$47milliontosupportNectaraspartoftheSuperScienceinitiativefinancedbytheEducationInvestmentFund(EIF),andsubsequentlyreceivedNCRISfunding.Nectarhasreceived$61millioningovernmentfunding,matchedbyco-investmentof$54millionfromAustralianuniversitiesandresearchorganisations

TheUniversityofMelbourneistheleadagentfortheadministrationofNectar.GovernanceofNectarisprovidedbytheNectarProjectBoard,chairedbyRussellYardley,with9othermembersdrawnfromparticipatingresearchinstitutions.

TheNectardirectoratecomprisestheequivalentof6.5full-timestaffandisledbytheDirector,AssociateProfessorGlennMoloney.

1Theroadmapfurtherstatesthat“anewcapabilityintheHumanities,ArtsandSocialSciences(HASS)hasbeenidentifiedinrecognitionofthewiderangingcontributionsthesedisciplinesmaketothenationalinterest.InvestmentinthisareawouldrelatetoaHASSeResearchinfrastructureincludingdatacreationanddigitisationofresearchmaterials”.TheHumanitiesNetworkedInfrastructure(HuNI)canbeviewedasaresponsetothis.

4

NectarisanexampleofasciencegatewaywhichShahand(2015)definesasa“web-basedenterpriseinformationsystemsthatprovidescientistswithcustomizedandeasyaccesstocommunity-specificdatacollections,computationaltoolsandcollaborativeservicesone-Infrastructures.”

Arecentreviewofsciencegateways(Barkeretal2017)describestheirbenefitsasfollows.

Sciencegatewaysareakeycomponentofthefuturedigitalresearchenvironment,enablingresearcherstoutilizeaglobalnetworkofinteractingdigitalplatformstoaccessandsharetheleading-edgedataandtoolsthatarecriticalfortheirresearch.Theybothfacilitate,andaresupportedby,broadermovementssuchasopenresearch,openscience,opensourcesoftwareandopendata.Consequently,sciencegatewaysarevaluabletoarangeofstakeholders:individualresearchers,researchcommunities,researchorganizationsandinstitutions(includingindustryandgovernment)andfundingagencies.

Definingsciencegatewaysintermsofcommoncharacteristicsandfunctionalityassistsinidentifyingtheirvaluetotheirstakeholders.Sciencegatewayslowerbarriersbyhidingthecomplexityoftheunderlyingdigitalresearchinfrastructureandsimplifyingaccesstobest-practicetools,dataandresources,therebydemocratizingtheirusage.Theycanenablecollaborationandbuildcommunities,sharingdataandanalysesamongmultidisciplinaryandgeographicallydispersedresearchgroups,leadingtoincreasedopenness.Theycanenableresearchtobeundertakenmoreefficientlythroughtheprovisionofmodellingandothersoftwareandhardwareresourcesthroughasingleportal,andenableresearchtobeundertakenthatwouldnototherwisebeconducted.Researchersnolongerneedtobephysicallyco-locatedbecauseresourcescanbegloballydistributed,andthisalsoenablesinclusionoflessadvantagedresearchers/institutions.Bysharingresourcesacrossmultipleinstitutions,thecostsofsettingupandsupportingresearchinfrastructureislowered,aseachinstitutionisnolongerrequiredtosupportareplicaofdata,computeandtoolsattheirsite.Forgatewaysthatareopensource,theirverybuildingandevolutioncanbedemocratized.Anycommunitymembercandownloadandusecodeandalsocontributefeatures,forexample,viagitpullrequests.

Nectarprovidesthemajorityofitsfundingthroughtwomaininvestmentprogramsine-researchinfrastructure:

• VirtualLaboratories• ResearchCloud

5

VirtualLaboratories

NectarVLsarerichdomain-orientedonlineenvironmentsthatdrawtogetherresearchdata,models,analysistoolsandworkflowstosupportcollaborativeresearchacrossinstitutionalanddisciplineboundaries.TheyarebuiltandledbytheAustralianresearchsectorandareusednationallyandinternationallybytheresearchcommunityandotherstakeholders,includingindustry.

Dataanalysisisincreasinglyanessentialaspectofresearchinmostacademicdisciplines,withlargeamountsofdatarequiringmorestorageandcomputingpowerthanmostdesktopenvironmentscanprovide.VirtualLaboratoriescanprovideHighPerformanceComputing(HPC)resourcestoanalyseandstoresuchdataandprovidetraininginadvancedanalyticalmethodsneededtoanalyselargedatasets.

NectarVLsarehostedonremoteserversandareaccessibleremotely,viatheinternet.Researchersnolongerneedtobephysicallyco-located:anyresources,includingpeople,canbegloballydistributed.Allthatisneededisaninternetconnectionforaccesstocollaborators,data,computationalandanalyticaltools.

Remoteaccesstoresearchresourcesincreasestheefficiencyofresearchandexpandsitsimpact.ThecostofHPCinfrastructurecanbeshared–itnolongerneedsnotbereplicatedineveryresearchfacilitythatrequiresit.ThesavingsincludecostlyupgradesthatareneededeveryfewyearstomaintainHPC.TheVLsplatformiswellsuitedfordeliveringtraininginadvancedanalyticalmethodstoupdateanalyticalskillsandtotrainanewgenerationofresearchers.Furthermore,VLsprovideaplatformforcollaboration,enablingresearcherstoshareideasandresearchoutcomeswithcolleaguesinAustraliaandaroundtheworld,acrossinstitutionalanddisciplineboundaries,ultimatelyexpandingtoourcollectiveknowledge.

OnceNectarwasestablisheditinitiatedtwoRequestsforProposalsforVLsinSeptember2010andApril2012.ContractswiththeinstitutionsthatproposedthesuccessfulVLsweresignedfromMay2012toJanuary2013.

VirtualLaboratoriesfundedunderNectarrecordedover19,000usersin2017across12VLs,including,onaverage,usersfromover20internationalorganizationsand30Australianorganizations(Barkeretal2017).

AcompletelistofVLsisgiveninAppendix1.

Sincetheirinception,accesstotheVLsandotherservicesfundedbyNectarhasbeenfreetousers.

ThefourVLsconsideredinthisstudyhavereceivedfundingfromNectarasshowninTable2.1.

6

Table2.1 FundingforfourVLsto2016-17,$

BCCVL CVL GVL HUNINectarcash to2013-14 1,236,063 1,618,108 2,334,121 1,329,0002014-15 220,000 320,000 445,000 193,0002015-16 230,000 225,000 230,000 192,5002016-17 177,000 177,000 177,000 120,000Total 1,863,063 2,340,108 3,186,121 1,834,500Partners Cash 600,000 In-kind 1,315,971 1,836,740 2,917,707 2,500,000 Total 3,779,034 4,176,848 6,103,828 4,334,500NectarCloud

NectarCloudprovidescomputinginfrastructure,softwareandservicesthatallowAustralia’sresearchcommunitytostore,access,andrundata,remotely,rapidlyandautonomously.NectarCloud’sself-servicestructureallowsuserstoaccesstheirowndataatanytimeandcollaboratewithothersfromtheirdesktopinafastandefficientway.

TheNectarResearchCloudisasinglenationalcloudcomputinginfrastructure,comprisedofsevencollaborating“nodes”.Nodesprocurehardwarenecessaryforrunningthecloudcompute,storageandnetworkservices.

Nectarisaworldleaderindeployinghighlyinnovativecloudcomputingtechnologyforthebenefitofresearch–providingopportunitiesforfederationwithemerginginternationalresearchclouds.TheNectarResearchCloudsupportstheNCRISmissiontodeliverworldclassresearchfacilitiessothatAustralianresearcherscansolvecomplexproblemsinAustraliaandinternationally.

7

3.Methodology

3.1Approachestoestimatingvalueandimpact

TheassessmentofprogramssuchasNectarVLs,whichprovidefreeaccesstoarangeofweb-basedinformationtechnologydatabases,analysissoftware,trainingandotherservices,isarelativelynewfieldandinvestigatorshaveusedavarietyofapproachestoestimatethevalueandimpactsoftheseprograms.

InarecentstudyofthevalueandimpactoftheEuropeanBioinformaticsInstitute(EBI),acentreforresearchandservicesinbioinformatics,whichispartofEuropeanMolecularBiologyLaboratory(EMBL).,BeagrieandHoughton(2016)reviewtheapproachestakenbystudiesofsciencefacilities.Theynotethatmostoftheseareoflargescaleinfrastructurefacilitiessuchassynchrotrons,andonlyafewofdatarepositoriesandrelatedinfrastructureandservices.TheyquoteareviewofstudiesbythegroupEvaRIO(EvaluationofResearchInfrastructuresinOpeninnovationandresearchsystems)(2013)whichfoundthatcost-benefitanalysisandtechniquesforestimatingthereturnonR&Dexpenditurewererelativelyrareintheestimationofthevalueandimpactofsciencefacilities.

BeagrieandHoughton(2016)foundthatthreemaintypesofanalysiswereusedinthesestudies:variousformsofinput-outputanalysis,casestudiesandexamples,andformsofcost-benefitanalysisusingactivitycostingand/orcontingentvaluationasthebasisoftheanalysis.

Input-outputanalysisisbestsuitedtoanalysisofsinglesitefacilitiessuchasasynchrotronwhilecasesstudiesarelimitedintheextenttowhichtheresultscanbescaleduporgeneralised.

BeagrieandHoughtonthereforeuseamixedmethodapproachtocost-benefitanalysisdrawingupontheirexperiencevaluingarangeofdataservicessuchastheNationalCrystallographyServiceatSouthamptonUniversityandtheUKDataArchiveattheUniversityofEssex(Beagrieetal.2010),theUKEconomicandSocialDataService(Beagrie,Houghton,Palaiologk,andWilliams2012),theUKArchaeologyDataServices(BeagrieandHoughton2013a),andtheBritishAtmosphericDataCentre(BeagrieandHoughton2013b).

Again,inmanyrespects,theservicesanalysedinthesestudiesaresimilartoNectarVLs.

InstudyingtheEBIandtheseotherservices,BeagrieandHoughton(2016)notethat,becauseoftheiropenaccessnature,theydonotrecordmuchinformationabouttheusersoftheservices,beyondwhatisavailablefromIPaddressesorwebdownloadswhenusersaccesstheseservices.Theynotefurtherthatmorecomprehensiveandreliabledatawouldimproveeconomicanalysisoftheseservices.

8

FollowingtheseauthorswehaveapproachedthetaskofmeasuringthevalueandimpactofNectarVLsinanumberofways.

TheseareshowninFigure3.1.Aswithmostlargesciencefacilities,theVLsprovidesvaluebothtotheuserandtothewidersociety.TheVLsprovideaservicetoacommunityofresearcherswhichcanbevaluedinaseveralways.Thesearediscussedinmoredetailbelowbutoneisgivenbythevalueofthetimespentusingtheserviceandanother,acontingentvaluation,isanestimateofitsmarketvaluei.e.thevalueauserwouldplaceontheserviceifitwasofferedinthemarketplace.InmeasuringthecontributiontoGDP,serviceswithnomarketvalue,areoftensimplyincludedatcost.Howeverevenwherethereisnomonetarytransactionitispossibletoestimatea‘market’valueofaservicetoaconsumer.Thismarketvalueis,theamounttheconsumerwouldeitherbewillingtopay(WTP)touseitoraccept(WTA)nottouseit.

Figure3.1 Impactvaluationmethodologies

Source:Beagrie,N.&Houghton,J.W.(2016)TheValueandImpactoftheEuropeanBioinformaticsInstitute(www.beagrie.com/publications).

ThiswillingnesstopayoracceptderivesfromtheefficiencygainseitherfromusingtheVL,orforgonefromnotusingit,respectively.TheseefficiencygainscanbeseparatelyestimatedfromdatacollectedfromVLusersandprovidesacredibilitycheckontheWTPandWTAestimates.

Beyondthesesourcesofvalue,liethereturnstoR&DwhicharederivedfromthevalueoftheresearchoutputsoftheusersoftheVLs.Thesearemoredifficulttoestimatebutincludesignificantsocialreturnsarisingfromspillovers,andotherbroadercommunitybenefits,

9

fromnewinnovativeproductssourcedinpartatleastfromthebenefitsgainedfromusingtheVLs.

1.Investmentandusevalue

Theinvestmentvalueofaproductistheamountofresourcesspentonitsproductionanddelivery.Theusevalueontheotherhandistheamountofresourcesspentbyusersonobtainingandusingtheproduct.MeasuresoftheamountoftimeandmoneyusedbyNectarVLusersinaccessingandusingtheVLservicesisanindicatoroftheminimumamountthattheserviceisworthtothem.

Informationabouttheresourcesinvolvedinthediscovery,access,anduseofNectarVLservicescanbeobtainedbyusersurveysandinterviews.

2.Contingentvaluation

Contingentvaluationinvolvestheassignmentofmonetaryvaluestonon-marketgoodsandservicesbasedonpreferences(i.e.,PreferenceTheory).Itisoftenusedinthevaluationofenvironmentalandotherassets,theservicesofwhich,incontrasttogoodsandservicestradedinthemarketplace,areprovidedfreeoratnominalpricestousers.

TheusualmeasureofeconomicactivityisGrossDomesticProduct(GDP).Thisprovidesanaggregatevalueofgoodsandservicestradedinthemarketplaceincurrencyunits.Thevalueofthesetransactionsislargelyestablishedthroughapricesettledbydemandandsupplyoutcomes.However,therearewidelyacknowledgedshortcomingswithGDPasameasureofeconomicvalue(StiglitzCommission2008).Oneismeasuringthevalueofservices.

Services,particularlythoseprovidedbythepublicsector,areoftenincludedatcostbecausethereisnodirectdemandandsupplyinteractionbetweenbuyersandsellers.Ineconomicterms,thereisnomechanismwherebytheconsumerisabletorevealherpreferencesofhowmuchshewouldbewillingtobuyataparticularprice.ThisoftenmeansthatservicesareundervaluedinestimatingGDP,becausetheuserswouldbewillingtopaysignificantlymorethanthecostofproductioniftheyweregiventhatopportunity.ThisislikelytobethecasefortheNectarvirtuallabs.

ThevalueoftheservicesprovidedbytheVLswillbeincludedinGDPlargelybasedonthesalarycostsofoperatingthelabs,wheninfact;thevalueplacedontheservicesbyusersmaybesignificantlygreater.Thecontingentvaluationmethodologyprovidesonewayofestimatingthisvalue.

Ifagoodorservicecontributestohumanwelfare,ithaseconomicvalue,andwhethersomethingcontributestoanindividual’swelfareisdeterminedbywhetherornotitsatisfiesthatindividual’spreferences.Preferencesarerevealedbywhatanindividualiswillingtopayforagoodorserviceand/orbytheamountoftimeandotherresourcesspentobtainingthe

10

preferredgoodorservice.Wherepreferencesarenotrevealedinthemarket,individualscanbeaskedwhattheywouldbewillingtopayfor,ortoacceptinreturnforbeingwithout,thegoodorserviceinahypotheticalmarketsituation(i.e.,statedpreference).Forapublicgood,thevalueisthesumof“willingnesses”,asconsumptionisnon-rivalrous(i.e.,thesameinformationcanbeconsumedmanytimes).Thekeydifferencebetweenwillingnesstopayandwillingnesstoacceptisthattheformerisconstrainedbyabilitytopay(typicallybydisposableincome),whereasthelatterisnot.Hence,willingnesstopaydirectlymeasuresthedemandcurvewithabudgetaryconstraint,whilewillingnesstoacceptmeasuresthedemandcurvewithoutabudgetaryconstraint(BritishLibrary2004).

Thecontingentvaluationapproachinvolvesaskingwhatpeoplewouldbewillingtopay(toreceiveabenefitorremoveacost)oraccept(toforgoabenefitorincompensationforacost)ifamarketexistedfortheexternaleffect.(CommonwealthofAustralia2006)

3.Efficiencyimpacts

Widerbenefitsandimpactscanbeexploredbylookingattheefficiencygainsenjoyedbyusersandassigninganeconomicvaluetothem,suchasthevalueoftimesavings(productivity),andtheavoidanceofcostsforusersthatwouldotherwisebeinvolvedinthecreation/collectionofthedataforthemselvesorobtainingitelsewhere.Forthiswecombineusersurveyquestionsaboutperceivedefficiencyimpactswithactivitycosting.

4.Returnoninvestment

TherehavebeenanumberofstudiesbothinAustraliaandoverseasthathaveattemptedtomeasurethereturnstobothpublicandprivateinvestmentinresearchanddevelopment,andthecontributionthatR&Dandtechnologydevelopmentmorebroadlyhavemadetoeconomicgrowth.

TheProductivityCommissionhasundertakenanumberoflargestudiesontheroleofR&DinAustralia(e.g.IndustryCommission1995,ShanksandZheng2006).

AnanalysisofthereturnstohealthR&DbyAccessEconomics(2008)foundthathealthR&DprovidesreturnstoAustraliaof117%perannum,withabenefit-costratioof2.17.Inanextensionofthisstudyin2014.DeloitteAccessEconomics(2014)foundthateverydollarinvestedbytheMedicalResearchFutureFundwouldgeneratereturnsof$3.39infuturehealthandproductivitygains.

InexaminingtheseandotherAustralianandinternationalstudies,BeagrieandHoughton(2016)concludedthatthereturnstoresearchanddevelopmentweretypicallyintherangeof20%to60%perannum.IntheiranalysisbasedonamethodologydevelopedbyHoughtonandSheehan(2009)andHoughtonetal(2009)basedonamodifiedSolow-Swanmodelofeconomicgrowth,theyusedaconservativevalueof40%.

11

Theseandotherstudiesofhaveconsistentlyfoundlargereturnstoinvestmentinbothpublicandprivateresearchanddevelopment.

BalancedValueImpactModel

Inourdiscussionswiththe4VLsthatarethefocusofthisstudybothHuNIandtheGVLdrewourattentiontotheBalancedValueImpactModelwhichisanevaluationschemadevelopedbyProfessorSimonTannerofKing’sCollegeLondon(Tanner2012).Thispublicationisamanualforuserstoundertaketheseevaluationsandthemethodologyiscomplex.Fromabriefreviewoftheliteratureitdoesnotappeartohavebeenimplementedfullyinanyevaluationtodate.

WediscusstheBalancedValueImpactModelinmoredetailinSection7below.

3.2Surveydata

MostoftheapproachesoutlinedabovedependonhowusersuseVLservicesandthevaluethatcanbeputonthisuse.

AmajorlimitationexperiencedwithstudiesofthevalueandimpactofresearchinfrastructuresuchasNectarVLsisthepaucityofdatarelevanttotheapproachesoutlinedintheprevioussection.WhencommentingonthesestudiesBeagrieandHoughton(2016)notedthevariabilityofresearchinfrastructuremetricsandtheimplicationsforeconomicanalysisandquotedfromtheirreview(BeagrieandHoughton2014)asfollows

Itisalsoclearfromthesestudiesthatdifferentdatacentrescollectfinancialandoperationaldata,suchasuserstatistics,datadeposit,accessanddownloadstatistics,tovaryinglevelsofdetailandusingdifferentdefinitions.Moreguidanceisneededonthecollectionofsuchdata.Doingsowouldhelptoensureagreaterdegreeofstandardisationofoperationalrecordsacrossdatacentres.Thiswouldbeofgreatestbenefittofundersinvestinginarangeofdatacentres,andwouldprovidemorecomprehensiveandreliabledataforeconomicanalysis.Therewouldbeconsiderableadvantagetoprovidingguidanceregardingthecollectionofsuchdataasitisfundamentaltotheeconomicanalysisandinmakingthebusinessorfundingcase.

Funderswillneedtoensureallowanceismadeinbudgetstoenablesuchcentrestocollectadequateinformationsothatproperevaluationscanbeundertaken.

ToaddressthepotentiallimitationsofusagedatacollectedbyNectarVLs,andtoprovidedatarelevanttotheapproachesoutlinedabove,anonlinesurveyquestionnairewasdevelopedtomeasureaspectsoftheuserexperiencewithVLsusingtheQualtricson-linesoftwaretool.(https://www.qualtrics.com/homepage/)

ThestartingpointforthequestionnairewasthatusedbyBeagrieandHoughton(2016)butmodifiedtoaddressthesituationofeachVL.Draftinitialsurveyquestionnaireswere

12

developedbyVISESanddiscussedwithkeyNectarandVLpersonnel.Theywerethenrefinediterativelyoveranumberofdiscussioncyclesandpilottested.

Thequestionnaireusedarangeofstandardsurveyapproaches,includingtheuseof“criticalinstances”,suchasthelastdataaccessed/downloaded(forusers).Anumberofquestionssoughtspecificinformationon:thetimeandcostofaccessforusers;thebenefitsandefficiencyimpactsofaccess;andcontingentvaluation(i.e.,willingnesstopayoraccept)usingstatedpreferencetechniques.Answerstothesequestionswereinterpretedcarefully,inthecontextofopen-endedtextcommentsinthesurvey.ThesequantitativequestionsweresupplementedbyqualitativequestionsaskingforviewsontheimportanceandimpactoftheVLforusers,toensurethatthequantitativeandqualitativefindingswereinaccord.

ThesurveyswereprogressivelyimplementedfromApriltoMay2017.

13

4.TheVirtualLaboratories

4.1BiodiversityandClimateChangeVirtualLaboratory(BCCVL)

TheBiodiversityandClimateChangeVirtualLaboratorywasformedunderanagreementinNovember2012betweenNectarandGriffithUniversity.TheothermainpartnersinBCCVLare

• JamesCookUniversity• UniversityofNewSouthWales• MacquarieUniversity• UniversityofCanberra• AtlasofLivingAustralia(ALA),ane-infrastructurefundedbyNCRIS• TerrestrialEcosystemResearchNetwork(TERN),anationalobservatoryfor

Australianecosystems,deliveringdatastreamsthatenableenvironmentalresearchandmanagement,withfundingfromNCRIS

• QueenslandCyberInfrastructureFoundation(QCIF)aHPCconsortiumofQueenslanduniversities

Undertheagreement,Nectarhasprovidedfundingof$1,863,063to2016-17andpartnershavecommittedto$600,000incashand$1,315,971in-kind.TheBCCVLwebsitewaslaunchedinAugust2014.

TheBCCVLprojectteamof8peopleislocatedatGriffithUniversityandheadedbyMrHamishHolewa.IthasaSteeringCommitteeof17peopledrawnfromparticipatinginstitutionsandNectar,aswellasanEcologicalModellingScientificAdvisoryGroupof8people.

TheBiodiversityandClimateChangeVirtualLaboratory(BCCVL)isa“onestopmodellingshop”thatsimplifiestheprocessofbiodiversity-climatechangemodelling.ItsmissionistoconnecttheresearchcommunitytoAustralia’snationalcomputationinfrastructurebyintegratingasuiteoftoolsinacoherentonlineenvironmentwhereresearcherscanaccessdataandperformdataanalysisandmodelling.

Previously,thelinesofinquiryintobiodiversityandclimatechangeimpactswerestymiedduetoresearchers’inabilitytoaccessastandardisedsetoftoolsforanalysisandrequisitedatasources,andcomputationallimitations.

ThegoaloftheBCCVListointegratethesetoolsanddatasetswithhigh-performancecomputersandmajordatastoragefacilities,therebyenablingmoreefficientinvestigationofbiologicalsystemsandfacilitatingthedevelopmentofadditionalresearchtrajectoriescurrentlynotpossibleduetocomputationallimitations.

14

CentraltothemodellingfacilitatedbyBCCVLisspeciesdistributionmodelling,alternativelyknownasenvironmentalnichemodelling,(ecological)nichemodelling,predictivehabitatdistributionmodelling,orclimateenvelopemodelling.Itreferstotheprocessofusingcomputeralgorithmstopredictthedistributionofspeciesingeographicspaceonthebasisofamathematicalrepresentationoftheirknowndistributioninenvironmentalspace(=realisedecologicalniche).Theenvironmentisinmostcasesrepresentedbyclimatedata(suchastemperature,andprecipitation),butothervariablessuchassoiltype,waterdepth,andlandcovercanalsobeused.Thesemodelsallowforinterpolatingbetweenalimitednumberofspeciesoccurrenceandtheyareusedinseveralresearchareasinconservationbiology,ecologyandevolution.

Developingaspeciesdistributionmodelbeginswithobservationsofspeciesoccurrences:theseareplaceswhereweknowaspecieshasbeenfound.Theseoccurrencesaremostlypoint-basedandcomefromsourcessuchasmuseumrecordsandobservationsofexpertsinthefield.BCCVLuserscanuploadtheirownspeciesdistributiondataoraccessoneormoreofthemanydatasetsprovidedbyBCCVL.

Tocalibrateacorrelativespeciesdistributionmodeltwotypesofinputdataareneeded:speciesoccurrences,andmeasurementsofasuiteofenvironmentalvariables,suchastemperatureandrainfall.Thesetwotypesofdataarethenputintoanalgorithmtofindassociationsbetweentheknownoccurrencesofaspeciesandtheenvironmentalconditionsatthosesites,toidentifytheenvironmentalconditionsthataresuitableforaspeciestosurvive.Thisprovidesinformationaboutwherespeciesoccurandsomethingabouttheenvironmentalconditionsofthoseplaces.Thealgorithmusesthesetwotypesofinformationtoestimatetheprobabilityofaspeciesoccurringinaplaceassomefunctionoftheenvironmentalconditionsofthatplace.

BCCVLcurrentlysupports17differenttypesofspeciesdistributionalgorithms.

BCCVLalsooffersarangeofclimatechangemodelsthatcanbeusedinconjunctionwithspeciesdistributionmodellingtoestimatetheimpactofclimatechangeonspeciesdistribution.

TheBCCVLwebsiteoffersaKnowledgeBasewhichcanguideusersthroughthemodellingprocessbyprovidinginformationaboutthestepsinvolvedinmodellingandthedatabasesandotherresourcesavailable.

Thewebsitealsooffersatrainingcourseof10modulesexplainingSDM,thechoiceofalgorithmsandhowtointerpretmodellingresults.

BCCVLoffersworkshopsaimedatacademicsandindustryprofessionals,forinstanceHDRstudents,environmental/climatescientistsandresearchers,ecologists,decision-makers,membersofgovernmentandindustrygroups.Italsoprovidesworkshopsforundergraduatestudentsstudyinginareassuchasecology,environmentalscience,sustainability,climate

15

changeimpacts,biology,floraandfauna,animalbehaviour,planninganddevelopment,andconservation.

4.2CharacterisationVirtualLaboratory(CVL)

TheCharacterisationVirtualLaboratory(CVL)wasformedunderanagreementinMay2012betweenNectarandMonashUniversity.FoundingpartnersinCVLare

• AustralianMicroscopyandMicroanalysisResearchFacility• AustralianNuclearScienceandTechnologyOrganisation• AustralianSynchrotron• NationalImagingFacility• AustralianNationalUniversity• UniversityofSydney• UniversityofQueensland

Throughitslife,anumberofprojectpartnershavejoinedtheCVLandengageddirectlythroughprojectfunding:

• CSIRO• DeakinUniversity• Intersect,anditspartners• QCIF,anditspartners• RMIT• ThePawseySupercomputingCentre• TheTerrestrialEcosystemResearchNetwork• UniversityofMelbourne• UniversityofNewSouthWales• UniversityofWesternAustralia• VicNode,anditspartners

Undertheagreement,Nectarhasprovidedfundingof$2,340,108to2016-17andpartnershavecommittedto$1,836,740in-kind.

TheCVLwebsitewaslaunchedinMarch2013.

ThepredominantpractiseoftheCVLprojectistoworkwithinstrumentfacilitiestoprovidedatacapture,analysisandvisualisationservicesandtherebyunderpinthefacilityusercommunity.

CVLhasworkedwithorisworkingwith26facilitiestointegrateover100instrumentswithatotalvalueofaround$250million..AcompletelistofthesefacilitiesisgiveninAppendix2.

16

TheCharacterisationVirtualLaboratory(CVL)aimstointegrateAustralia’simagingequipmentwithspecialisedHighPerformanceComputingcapabilitiesandwithdatacollectionnodesandprovidescientistswithacommonenvironmentforanalysisandcollaboration.Itinitiallydevelopedfourresearchapplication(‘drivers’)inmulti-modalorlarge-scaleimaginginneuroscience,structuralbiology,atomprobeandX-rayscience,andthroughpartnership,thishasbeenextendedtocytometry,neutron-beamimaging,lightmicroscopy,andbioinformatics.

Eachdriverisledbyaworld-classresearchgroup,issupportedbyanAustralianresearchconsortiumandisinanationalresearchpriorityarea.TheresultsfromthisdevelopmentaredistributedtothecommunitythroughCVL“Workbenches”.

CVLusershavetheoptionofusingthecomputingresourcesontheNectarresearchcloudoraccessingMASSIVE(Multi-modalAustralianScienceSImagingandVisualisationEnvironment)theHPCfacilityjointventurebetweenMonashUniversity,CSIROandtheAustralianSynchrotron

AnimportantroleforCVLhasbeentoestablishthecapabilitywithinimagingsitesforautomaticcaptureandstorageofimagingdatatothecloud.ThisenablesuserstobeabletoaccesstheirimagingdatathroughtheCVLwebsiteandtoperformanalysisusingthesoftwaretoolsprovidedbyCVLbasedaroundtheMyTardisandStore.Monashtools(Ceguerraetal2013).

CVLhasdevelopedanumberofapplicationsthatarereusedacrossarangeofAustralianfacilities.Theseinclude:extensionstotheMyTardisdatamanagementplatformtosupportinstrumentfacilities;aninstrumentintegrationapp,calledMyDatatomakeintegrationquicker,simplerandlessreliantonspecialistITsupport;andStrudel,atoolthatmakesaccessingremoteanalysisenvironmentseasier.

4.3GenomicsVirtualLaboratory(GVL)

TheGenomicsVirtualLaboratorywasformedunderanagreementinMay2012betweenNectarandtheUniversityofQueensland.TheothermainpartnerinGVListheUniversityofMelbourne.ThefollowingorganisationsarepartnersorcollaboratorswithGVL

• TheUniversityofQueensland• QFABBioinformatics• QueenslandCyberInfrastructureFoundation• MelbourneBioinformatics(formerlyVictorianLifeSciencesComputationInitiative)

VictorianeResearchStrategicInitiative• TheUniversityofMelbourne• BakerIDIHeartandDiabetesInstitute• PeterMacCallumCancerCentre

17

• TheGarvanInstituteofMedicalResearch• VictorChangCardiacResearchInstitute• CSIRO• BioplatformsAustralia• MonashUniversity• TheUniversityofSydney• TheUniversityofWesternAustralia• AustralianBioinformaticsNetwork• EMBLAustralia• AustralianGenomeResearchFacility• AustralianNationalDataService

Undertheagreement,Nectarhasprovidedfundingof$3,186,121to2016-17andpartnershavecommittedto$2,917,707in-kind.

TheGVLisadministeredbyMelbourneBioinformatics.Ithasastaffof8ledbyAssociateProfAndrewLonie,asDirectorofMelbourneBioinformatics.

TheGenomicsVirtualLaboratoryprovidesacloud-basedsuiteofgenomicsanalysistoolsforlifescienceresearchandtraining.

Biologistswithoutcomputersciencetrainingcangostraighttoauser-friendlyplatformwhichhostsasuiteoftestedbioinformaticstoolsandpipelinesforfast,consistent,dataanalysis.Theplatformisconstantlyupdatedtohavethelatestfeaturesinusebyexpertbioinformaticians.Adoptedbothlocallyandoverseas,theGVLhasalreadybeenrecognisedasaqualityplatformtohelpaddresstheshortageofbioinformaticsexpertisearoundtheworldandmanagethecomplex,multiple-layereddataanalysistasksconfrontinglifescientiststoday.

Nationallyandinternationallyitisbeingusedbothbylifescientistsworkingwithgenomicdataandacademicsteachingbioinformaticsatundergraduateandpost-graduatelevels.

LifescientistswithoutaccesstobioinformaticsexpertisearetheprimaryusersoftheGVL.Practisingbioinformaticianswhoknowhowtofindtherighttoolforthedataanalysisjob,findthattheGVLisworkingfortheminotherways.SmallbioinformaticsgroupsorlonepractitionersusetheGVLtotraintheirlocalteamstodotheirownsimplebioinformaticstasksontheGVL,freeingupcapacitytoworkonmorecomplexresearchproblemsortocollaboratemorebroadly.TheGVLwebsiteisbuiltaroundGalaxy,anopen,web-basedplatformfordataintensivebiomedicalresearch.GalaxyisaninternationalcollaborativeprojectmanagedbytheCenterforComparativeGenomicsandBioinformaticsatPennStateUniversity,andtheDepartmentofBiologyatJohnsHopkinsUniversity.

18

UsersofGVLhavetheoptionoflaunchingtheirownprivateserveronapublicorprivatecloud,withtheGVLpre-installed,oraccessingitthroughoneofthepublicGVLservicesontheGVLwebsitemanagedbyGVL.MostuserswithoutrelevantITskillsuseoneofthreemainpublicservicesnamely

• GalaxyMelbourne• GalaxyQueensland• GalaxyTut

GalaxyTutisusedfortrainingpurposes.Aswell,theGVLprovideson-linebasicandadvancedGalaxytutorialsandcommandlinetutorialswhichgivestepbystepinstructionsforanexampleanalysis.Protocolsoutlinetheanalysismethodsandsuggestandcomparetoolsforeachstepratherthanprescribethem.

GVLuserstypicallyloadgenomicsequencesthatarethenmanipulatedandanalysedusingacomprehensivesuiteoftools.

AsidefromthepublicservicesmanagedbytheGVL,anumberofotherinstitutionssuchasCSIROandLaTrobeUniversityhavedownloadedtheGVLtotheirserverstolocallymanageaccessfortheirinstitutionalusers.

4.4HumanitiesNetworkedInfrastructureVirtualLaboratory(HuNI)

TheHumanitiesNetworkedInfrastructureVirtualLaboratory(HuNI)wasformedunderanagreementwithDeakinUniversityinMay2012.Nectarhascontributedfundingof$1,834,500to2016-17andpartnershavecommittedto$2,500,000in-kind.

ThepartnersinHuNIhaveincluded

• AustralianInstituteofAboriginalandTorresStraitIslanderStudies(AIATSIS)• AustralianNationalUniversity• DeakinUniversity• FlindersUniversity• IntersectAustralia• MacquarieUniversity• RMITUniversity• UniversityofMelbourne• UniversityofNewSouthWales• UniversityofQueensland• UniversityofSydney• UniversityofWesternAustralia• V3Alliance• NationalLibraryofAustralia

19

• AustralianCentrefortheMovingImage(ACMI)

TheProjectDirectorofHuNIisProfessorDebVerhoevenatDeakinUniversity.

GoogleAnalyticsdataprovidedbyHuNIindicatesaround4,600usersofHuNIduring2016.

HuNI(HumanitiesNetworkedInfrastructure)combinesdatafrommanyAustralianculturalwebsitesintoalargehumanitiesandcreativeartsdatabase.HuNIdatacoversalldisciplinesandbringstogetherinformationaboutthepeople,works,events,organisationsandplacesthatmakeupthecountry'srichculturallandscape.

HuNIcombinesinformationfrom32ofAustralia’smostsignificantculturaldatasets.Thesedatasetscomprisemorethan17millionauthoritativerecordsrelatingtothepeople,organisations,objectsandeventsthatmakeupAustralia'srichculturalheritage.HuNIalsoenablesresearcherstoworkwithandsharethislarge-scaleaggregationofculturalinformation.HuNIhasbeendevelopedasapartnershipbetween16publicinstitutions,ledbyDeakinUniversity.

AccesstotheHuNIwebsitewasfirstavailableinJuly2012andwasofficiallylaunchedinOctober2014.

AsVerhoevenandBurrows2015suggeststhisplacesHuNI

‘somewherebetweena“datawarehouse”inwhichtheincomingdataarefirstcleanedandorganisedintoaconsistentschemaanda“datalake”inwhichtheincomingdataareingestedintheirrawformandtheresponsibilityormakingsenseofthedataliesentirelywiththeenduser’p418

Theconstituentdatabases,havebeenreconfiguredbyHuNIsothattherecordsaremappedtosixcoreentities:Person,Organization,Event,Work,Place,andConcept.(VerhoevenandBurrows2015).

Theuserbeginstheirsearchforanitemofinterestinoneoftheseentities.Thatcouldbeanameofapersonoraparticularorganisationoraconcept(topic)ofinterest.ThepowerofHuNIistoretainasdatawithinHuNItheconnectionsdiscoveredbetweenitemsintheentities,suchasaconnectionlinkingapersontoothersand/oranorganisationandaconceptetc.madebytheuser.Thesearesavedbytheuserandmaybeplacedinthepublicdomainandthusmadeavailableforotherusers.Subsequentusersmaydisputethelinksmadeorsupplementthelinks,creatingarichersetofnetworksconnectedtooneormoreoftheentities.

Anexampleofsuchanetwork(aHuNIKnowledgeGraph)isshowninFigure4.1.

Thisgraphshowslinksrelatedtothecollection:AustralianFashion1850-1950,includingrelatedpersonsandworks.InparticularitlinkstheAustralianFashionCollectionviaa1938

20

filmwithafashiontheme,DadandDaveCometoTowntoacollectiongeneratedbyadifferentHuNIuserinvestigatingworksdirectedbyKenHallandproducedbyCinesoundProductions.TherearefurtherlinksmadetootherfilmsdirectedbyKenHall.BytracingthepathwayofthesevisuallinksaHuNIuserisabletoexpandtheirdiscoveryopportunitiesthroughtheserendipitousconnectionsprovidedbyamultitudeofHuNIusers.

Figure4.1 AnexampleofaHuNIKnowledgeGraph

SourceVerhoeven2016,p23

Asthefigurebelowillustrates,thisprocessconvertsisolateddataandinformationtoknowledgebycapturingtherelationshipsarisingfromtheexpertiseofHuNIusers.The

21

knowledgecreatedbytheseconnectionsisfurtherenrichedthroughthecollaborationofotherhumanitiesusersofthewebsite.

Figure4.2 ConnectionsbetweendataandinformationusingHuNI

SourceVerhoeven2016,p13

Thisfacilitationofanunstructuredapproachtoknowledgecreationwithinthedatabaseencouragesaserendipitousrelationshipsearchprocess.ThisisseenasamajoraddedvalueoftheVL,onethatisparticularlyapplicabletothehumanitiesstyleofresearchandonethatisnotcateredforbyconventionalarchivaldatabases.

‘HuNImovesbeyondthinkingofserendipityasonlyatechnicalproblemandinsteadtreatsitasamatterofsocial,philosophical,andpoliticalsignificance’(p23Verhoeven2016).

HuNIhasbeenestablishedwiththeideathatvaluableknowledgeandnewperspectivesareburiedintraditionalarchivaldatabases,whichcanbereleasedbythepartlyunanticipatedsearchprocessessupportedandusedbyHuNIusers(Verhoeven2016).PerhapsthiscreationofnewconnectionsisHuNI’shighestvalue.Itnotonlyfacilitatesthecreationofnewnetworksthroughlinkswithmultipledatabases,butalsohasthesoftwarethatallowstheconnectionsmadeinthesearchprocesstoberecorded,storedandmadeavailabletootherresearchers.OvertimethatarchiveoftheseHuNIKnowledgeGraphswilladdsignificantvaluetotheHuNIproject.

Atthisstagetheresourcesavailabletotheprojecthavebeenlargelydevotedtothedevelopmentofthesoftwarethatfacilitatesthecharacterisationoftheunderlyingarchivaldatabasesintermsofsearchableentities.Thefacilitytostoreuserdevelopedconnectionsisstillinitsinfancy.Anewsystemupgrade,includinganewuserinterface,isabouttobeinstalled,whichwillbetterfacilitatethecreation,storageandsearchingofvaluableHuNIKnowledgeGraphs.

22

HuNI’svalueliesintheincreasingparticipationofitsusers.ThemorelinksandcollectionsthataremadeinHuNI,themorevaluableatoolitbecomesforeveryone.ThismeansthatearlyonthebenefitversuseffortforHuNIusersisrelativelylow,thisrisesexponentiallyasmoreusersaddtheirexpertisetotheplatform.InvestmentinthepromotionofHuNIandbuildingofthecommunityisrequiredtoensureHuNImovespastthetippingpointofusers“gettingmoreoutofHuNIthantheyputin”.

23

5.DataonusageofNectarVirtualLaboratories

ThemethodologiesformeasuringthevalueandimpactofNectar’sVLsdependonbeingabletodescribetheextenttowhichtheVLshavebeenabletoattractusersoftheirservices.IntheevaluationswerelyondataprovidedbytheVLsonusageandontheresponsestotheon-linesurveysofusers.

AlthoughtheVLsdifferintheextentandtypeofdatatheycaptureaboutusageofVLservices,theyreadilyprovidedusagedatabasedontheregularcollectionstheymakeformonitoringpurposesandinresponsetofurtherrequestsfordifferentormoredetaileddata.Theirabilitytoprovidethisdatawaslimitedforsomebecauseitwasnotdatathatwasusuallycollectedinthecourseoftheiroperations.

ThetablesandfiguresbelowsummarisetheusageofVLservicesgenerallyfromwhentheybecameactivetothepresent.Forsome,veryearlydatawasnotavailableandthemostrecentdatawasforearly2017.

ItisclearthatthenumberofusersforaparticularVLincreasesrapidlyinthefirstfewyearsandhascontinuedtogrowoverthepastyear.

ThisstudyonthevalueandimpactofVLsisthereforeoccurringrelativelyearlyintheirlifetimes,presentingchallengesinassessingthelikelynumberofusersoverthenextfewyearsandthepotentialmaximumnumberofusers.ItisdifficulttoassesswhatpercentofthecurrentresearchbaserepresentsthemaximumachievablebyeachVLandwhenthiswilloccur.Itwouldbeexpectedthatoncethisoccurred,thefuturerateofgrowthwouldbelowerandjustreflectthenumbersofnewresearchersenteringtheresearchfieldsofrelevancetotheVL.

BCCVL

BCCVLprovidedmonthlyusagedatafromJune2015toMarch2017.FromthisdataweshowinTable5.1thenumbersofnewusersforeachcalendaryearandthecumulativenumberofusersatDecemberofeachyear.Tothisweaddestimatesofthenewusersandcumulativeuserscalculatedasfollows.

Overthe11monthstoMarch2017,thenumberofusersincreasedby3.75timescomparedtotheprevious11months.Ifitisassumedthatthenumberofnewusersin2017willbe2.0timesthatin2016,andthevaluein2018willbe1.5times2017,andwiththesamenumberofnewusersin2019and2020asin2018,thenthenumberofnewusersandthecumulativenumberofusersfor2017to2020willbeasinTable5.1.

ThesegrowthassumptionssuggestthatthecumulativenumberofusersbyDecember2020willbe11,246.

24

Table5.1 BCCVLactivitymeasures

Newusers

Cumulativeat

December2015 254 2542016 916 1,1702017 1,832 3,0022018 2,748 5,7502019 2,748 8,4982020 2,748 11,246CVL

CVLconsultedwitharangeofimagingfacilitiespartnersandprovidedestimatesofthenumberofusers.ToAugust2017thisamountedto2558,soweassumethatthenumberatDecember2016was2400.Basedonaninitialcohortof101atDecember2012,weinterpolatedthenumberofnewusersfrom2012to2016andestimatedthatthiswouldgrowby10%intheyearsto2020.Thisresultedinanestimateofthecumulativetotalof6,485bytheendofDecember2020(Table5.2).

Table5.2 CVLactivitymeasures

Newusers

Cumulativeat

December2012 101 1012013 300 4012014 500 9012015 700 1,6012016 800 2,4012017 880 3,2812018 968 4,2492019 1,065 5,3142020 1,171 6,485GVL

GVLhasprovidedusagedataforthethreemajorpublicserviceportals.Ithasnotbeenpossibletoobtainandactivitydataforthoseorganisationsandindividualsthathavedownloadedthedataandlaunchedtheirownprivateserveronapublicorprivatecloud,withtheGVLpre-installed.

GVLprovidedmonthlyusagedataforGalaxy–QueenslandfortheperiodDecember2013toFebruary2017.ThedataforGalaxy–MelbournecoveredSeptember2015toFebruary2017andforGalaxy–TutitwasDecember2012toFebruary2017.

25

ThenumbersofnewusersandthecumulativenumbersofusersatDecemberforthethreeportalsisshowninTable5.3.InasimilarmannertothatusedforBCCVL,weestimatethenumberornewusersandcumulativeusersto2020butrecognisethatthe3portalshavebeenoperatingfordifferentlengthsoftime.

ForGalaxy–QueenslandandGalaxy–Tutweassumea10%growthinnewusersforeachoftheyears2017to2020.ForGalaxy–Melbournethenumberofnewusersinthe9monthstoFebruary2017was2.6timesthatoftheprevious9months.Weassumethatthenumberofnewusersin2017willbe2.5timesthatof2016,thenumbersin2018willbe2.0timesthatof2017,thenumbersin2019willbe1.5timesthatof2018andthenumbersin2020willbethesameas2019.TheestimatednewusersandcumulativeusersatDecembereachyearareshowninTable5.3.

Table5.3 GVLactivitymeasures

Galaxy-Queensland Galaxy-Melbourne Galaxy-Tut Newusers Cumulative

atDecember

Newusers Cumulativeat

December

Newusers Cumulativeat

December2012 40 402013 27 27 117 1572014 285 312 1,060 1,2172015 586 898 65 65 510 1,7272016 631 1,529 408 473 587 2,3142017 694 2,223 1,020 1,493 646 2,9602018 764 2,987 2,040 3,533 710 3,6702019 840 3,826 3,060 6,593 781 4,4512020 924 4,750 3,060 9,653 859 5,311

26

6.EstimatingthevalueandimpactofNectarVirtualLaboratories

MostofthecalculationsreportedinthissectionrelyonestimatesofthetimespentbyusersaccessingVLservicesandthesalariesoftheseusers.

6.1Costdata

ForacademicuserswehavecalculatedthehourlyratefordifferentacademicpositionsbyobtainingsalarydatafromMonashUniversityandcalculatingthevaluefor2017.TheannualsalariesshowninTable6.1areaveragesofthelevelswithineachacademicgradeforthesepositions.WehavecheckedtheselevelswiththoseinanumberofotheruniversitiesandothersourcesofinformationandregardtheMonashUniversityvaluesasrepresentativeofsalarieswithinAustralianuniversities.Forcomparisonpurposes,thesalariesarearound5%higherthanthoseforVictoriaUniversity.Toobtainanhourlycostsequivalentwedividetheannualsalariesbytheaveragenumberofworkingdaysinayear(225)andbythenumberofhoursworkedperday(7.35).

FollowingBeagrieandHoughton(2016)weadd30%tothistoaccountfornon-wagelabourcosts.ThisisalsoinlinewithcurrentAustralianuniversityguidelines.TheindicativehourlycostscalculatedinthiswayareshowninTable6.1.

Forstudentandotherusersthestartingpointisthemedianstartingsalariesofbachelordegreegraduatesin2015publishedbyGraduateCareersAustraliaLtd.Theirmostrecentdataisfor2015.Toestimatethevaluesfor2017,weaddafurther4.4%,beingtheestimatedtwoyearincreaseinaverageweeklyearningsforfull-timeadults(ABS2017).

HourlycostsarecalculatedinthesamewayasforacademichourlycostsandtheseareshowninTable6.2.Fromthisweassumethatthehourlycostforstudentsisthehourlygraduatecostof$44.52

Table6.1 Academicsalaries,andhourlycosts

Level Salary,2017 Hourlysalary,2017

Hourlycosts,2017

$,000 $ $AcademicA 77,819 47.06 61.17AcademicB 102,469 61.96 80.55AcademicC 123,546 74.71 97.12AcademicD 145,211 87.81 114.15AcademicE 178,005 107.64 139.93

27

Table6.2 Annualgraduatesalariesandhourlycosts

Qualification Salary,2015 Salary,2017,est.

Hourlysalary,2017,est.

Hourlycosts,2017,est.

$,000 $,000 $ $AgriculturalScience 50.0 52.2 31.56 41.03BiologicalSciences 50.0 52.2 31.56 41.03ComputerScience 54.0 56.4 34.09 44.32EarthSciences 60.0 62.6 37.88 49.24Engineering 60.0 62.6 37.88 49.24Mathematics 60.0 62.6 37.88 49.24PhysicalSciences 50.0 52.2 31.56 41.03VeterinaryScience 50.0 52.2 31.56 41.03Average 54.3 56.6 34.25 44.52 TOTAL(all

qualifications)54.0 56.4 34.09 44.32

Source:Table1:Medianstartingsalariesofbachelordegreegraduatesinfirstfull-timeemploymentandagedlessthan25,byfieldofeducationandsectorofemployment,2015($,000,n)¤†,GraduateCareersAustraliaLtd,GraduateSalariesReport2015,GraduateCareersAustraliaLtd,2016athttp://www.graduatecareers.com.au/research/researchreports/graduatesalaries/

Intheon-linesurveyofusers,participantswereaskedabouttheirpositionwithintheirorganisation.Allocatingbothacademicandgraduateleveltothesepositionsandusingthedistributionofanswersfromrespondents,wecalculatedameanhourlycostofusersbasedonthehourlycostsinTables6.1and6.2.

ForBCCVL,CVLandGVLthehourlycostofuserswere$74.82(or$63,127annually),$66.39($56,015annually)and$71.02($59,925annually)respectively.

Investmentvalue

IntheirstudyofEBI,BeagrieandHoughtonwereunabletoestimatethetotalvalueofallthecomponentsofEBI’sinvestmentvalue,suchasdataacquisition,depositingdata,collaborationcostandaddingvaluetothedata.Theyreliedthereforeonanestimateoftheaverageannualoperatingexpenditureasaproxy.

ThesetupandoperatingcostsofeachVLtotheyear2016-17weredescribedinTable2.2inSectionearlier.

IfweassumethattheannualoperatingexpensesofeachVLisequaltotheaverageoftheyears2014-15to2016-17asshowninTable2.2andweapplythattoeachyearoutto2020,wecancalculatetheestimatedaccumulatedcostofeachVLto2020.Thisisshownintable6.3andistheestimatedinvestmentvalueofeachVL.

28

DividingthisbythenumberofyearsfromVLinceptionto2020,wecancalculatetheannualvalueoftheoverallinvestmentmadeineachVL.AscanbeseenfromTable6.3thisisintherange$550,000to$600,000forBCCVL,CVLandHuNIandissomewhathigheratabout$870,000forGVL.

Table6.3 EstimatedtotalandannualcostforVirtualLaboratories

BCCVL CVL GVL HUNITotalcoststo2016-17 3,779,034 4,176,848 6,103,828 4,334,500Averageannualoperatingcost 209,000 240,667 284,000 168,500Totalcoststo2020 4,406,034 4,898,848 6,955,828 4,840,000Annualvalueoftotalcoststo2020 550,754 612,356 869,479 605,000

Contingentvaluation

Thecontingentvalueofanon-marketgoodorserviceistheamountusersarewillingtopayforitand/orarewillingtoacceptinreturnforgivingitup.Forapublicgoodthevalueisthesumofwillingnesses,asconsumptionisnon-rivalrous(e.g.,thesameinformationcanbeconsumedmanytimes).Thekeydifferenceisthattheamountthatusersarewillingtoacceptinreturnforgivingupaccessistypicallyhigherthantheamounttheywouldbewillingtopay,primarilybecausethelatterisconstrainedbywhattheycanafford(e.g.,bydisposableincome,limitedresearchgrants,etc.).

BCCVL

ForBCCVL,themeanamountthatuserswerewillingtopayforaccesstoBCCVLserviceswas$1,154(medianvalue$250).OntheotherhandthemeanamountuserswouldbewillingtoaccepttoforegoBCCVLserviceswas$10,005(medianvalue$5,000).UsersnaturallyputahighervalueonBCCVLserviceswhentheyarenotsubjecttoanincomeconstraint.Asacomparison,BeagrieandHoughtonfoundthemeanwillingnesstopayforEBIwasaround$2,800(GBP1628).

Usingthemeanwillingnesstopayvalueof$1,154,thecontingentvaluationofBCCVLbyuserswouldthereforehavebeen$1,350,180intheperiodtoDecember2016,oranannualequivalentof$852,745,risingto$13.0millionbyDecember2020,or$2.3millionperyear.(Table6.4).TheannualequivalentforeachyeariscalculatedbydividingthecumulativevaluetothatpointbythenumberofmonthselapsedsinceinceptionoftheVLandmultiplyingby12.AsusageofBCCVLbeganinJune2015,theannualvalueestimatedfor2015ishigherthanthecumulativevalue.

Usingthehighervaluemeasuredbyusers’willingnesstoaccept,thecontingentvaluationofBCCVLwouldbe$112.5millionbyDecember2020or$20.1millionperyear.

29

Table6.4 BCCVLcontingentvaluation

Cumulativeusers

WTP WTA

Cumulative Annual Cumulative Annual2015 254 293,116 502,485 2,540,000 4,354,2862016 1,170 1,350,180 852,745 11,700,000 7,389,4742017 3,002 3,464,308 1,341,022 30,020,000 11,620,6452018 5,750 6,635,500 1,851,767 57,500,000 16,046,5122019 8,498 9,806,692 2,139,642 84,980,000 18,541,0912020 11,246 12,977,884 2,324,397 112,460,000 20,142,090CVL

ThemeanamountthatCVLuserswerewillingtopaywas$1,524(median$900),whiletheirmeanwillingnesstoacceptwas$14,130(median$20,000).

Usingthemeanvalueof$1,524andtheestimatesofCVLusersabove,suggeststhatbyDecember2020thecontingentvalueofCVLwouldbe$9.9millionoranaverageannualvalueof$1.2million.Usingthewillingnesstoacceptvaluethiswouldbe$91.6millionbyDecember2020,or$11.5millionannually(Table6.5).

Table6.5 CVLcontingentvaluation

Cumulativeusers

WTP WTA

Cumulative Annual Cumulative Annual2013 301 611,124 611,124 5,666,130 5,666,1302014 601 1,373,124 686,562 12,731,130 6,365,5652015 1,051 2,439,924 813,308 22,622,130 7,540,7102016 1,550 3,659,124 914,781 33,926,130 8,481,5342017 2,100 5,000,244 1,000,050 46,360,530 9,272,1062018 2,705 6,475,476 1,079,246 60,038,370 10,006,3952019 3,371 8,098,231 1,156,890 75,083,994 10,726,2862020 4,103 9,883,262 1,235,410 91,634,180 11,454,271GVL

ThemeanamountthatGalaxy-Melbourneuserswerewillingtopaywas$606(median$100),whiletheirmeanwillingnesstoacceptwas$1285(median$800).

Usingthemeanvalueof$606andtheestimatesofallGVLusersabove,suggeststhatbyDecember2020thecontingentvalueofGVLwouldbe$11.9millionoranaverageannualvalueof$1.9million.Usingthewillingnesstoacceptvaluethiswouldbe$25.3millionbyDecember2020,or$4.0millionannually(Table6.6).

30

Table6.6 GVLcontingentvaluation

Cumulativeusers

WTP WTA

Cumulative Annual Cumulative Annual2013 184 135,744 284,167 287,840 602,5662014 1,529 926,574 528,529 1,964,765 1,120,7262015 2,690 1,630,140 718,806 3,456,650 1,524,2002016 4,316 2,615,496 858,905 5,546,060 1,821,2762017 6,676 4,045,535 1,070,515 8,578,403 2,269,9862018 10,190 6,174,885 1,363,932 13,093,610 2,892,1662019 14,871 9,011,667 1,684,004 19,108,898 3,570,8672020 19,714 11,946,691 1,901,364 25,332,505 4,031,769

Efficiencyimpacts

Intheon-linesurveys,userswereaskedthefollowingquestionsabouttheamountoftimetheyspentdoingresearch,theshareofthattimeworkingwithdataandtheirestimatesofthetimesavingsfromusingtheVL.

Overthelasttwelvemonths,onaveragehowmanyhoursperweekdidyouspendonresearch?

Canyouestimatetheapproximateshareofyourtotalresearchworkingtimespentwithdataduringthelasttwelvemonths(e.g.creating,manipulatingandanalysingdata)?

Alldata:

approximatepercentofmytotalresearchworkingtime

DatafromGVL:

approximatepercentofmytotalresearchworkingtime

Towhatextent,ifany,hasyouruseofGVLservicesandresourceschangedyourresearchefficiency(i.e.thetimesavedcomparedtothesituationifGVLdidnotexist?

ThetimespentonresearchexpressedinhoursperweekisshowninTable6.7foreachVL.

UsingtheaverageannualcostperusercalculatedinSection6.1,thevaluesinTable6.7canbeusedtocalculatetheannualcostperuserof(i)thetimespentonresearch,(ii)thetimespentworkingwithdata,(iii)thetimespentworkingwithVL,and(iv)theresultingincreaseinresearchefficiency.TheresultsofthesecalculationscanbeseeninTable6.7.TheaverageannualvalueperuseroftheefficiencyimpactofBCCVL,CVLandGVLare$11,898,$13,240and$23,431respectively.Asacomparison,BeagrieandHoughtonfoundthatEBIwouldbeworthGBP26,000or$44,828perpersonperannum.

31

ApplyingthesevaluestothecumulativenumberofusersforeachVLweestimatethattheefficiencyimprovementduetoeachVLisasshowninTable6.9.Moreconservatively,ifweapplythesevaluesjusttothenumberofnewuserseachyearforeachVL,thecalculatedvaluesareasshowninTable6.10.WhichevermethodisuseddemonstratesthattherearesubstantialbenefitsinresearchefficiencyarisingfromtheVLs.

Table6.7 Surveyresponsesonresearchandefficiency

BCCVL CVL GVLTimespentonresearch,hoursperweek mean 22.8 28.5 37.9 median 20.0 30.0 40.0Shareofresearchtimeworkingwithdata,% mean 65.1 59.9 66.4 median 70.0 50.0 75.0ShareofresearchtimeworkingwithVL,% mean 11.8 28.0 11.4 median 5.0 23.0 7.0Increaseinresearchefficiency,% mean 31.0 31.1 39.1 median 30.0 30.0 50.0

Table6.8 Averageannualvalueofefficiencyimpactperuser

BCCVL CVL GVLAnnualcosts 63,127 56,015 59,925 Annualcostoftimespentonresearch mean 38,381 42,572 59,925 median 33,668 44,812 59,925Annualcostofresearchtimeworkingwith

datamean 24,986 25,500 39,790

median 23,567 22,406 44,944AnnualcostofresearchtimeworkingwithVL mean 4,529 11,920 6,831 median 1,683 10,307 4,195Increaseinresearchefficiency mean 11,898 13,240 23,431 median 10,100 13,444 29,963

Table6.9 ValueofVirtualLaboratoriesonresearchefficiency,usingcumulativeusers,$

Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 5,309,154 4,311,244 0 5,309,154 4,311,2342014 0 11,929,046 35,825,502 0 5,964,523 17,912,7072015 3,022,137 21,196,896 63,028,516 1,007,386 7,065,632 21,009,4542016 13,920,867 31,788,724 101,126,793 3,480,241 7,947,181 25,281,6362017 35,718,327 43,439,735 156,423,186 7,143,715 8,687,947 31,284,5602018 68,414,518 56,255,846 238,758,578 11,402,498 9,375,974 39,792,9992019 101,110,708 70,353,569 348,437,568 14,444,486 10,050,510 49,776,673

32

2020 133,806,898 85,861,065 461,912,327 16,725,977 10,732,633 57,738,899

Table6.10 ValueofVirtualLaboratoriesonresearchefficiency,usingnewusers,$

Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 3,971,935 3,374,009 0 3,971,935 3,374,0092014 0 6,619,892 31,514,180 0 3,309,946 15,757,0902015 3,022,158 9,267,849 27,202,947 1,007,386 3,089,283 9,067,6492016 10,898,805 10,591,828 38,098,184 2,724,701 2,647,957 9,524,5462017 21,797,610 11,651,011 55,296,257 4,359,522 2,330,202 11,059,2512018 32,696,415 12,816,112 82,335,190 5,449,402 2,136,019 13,722,5322019 32,696,415 14,097,723 109,678,720 4,670,916 2,013,960 15,668,3892020 32,696,415 15,507,495 113,474,480 4,087,052 1,938,437 14,184,310

AdditionalresearchmadepossiblebyVirtualLaboratories

Theon-linesurveyaskedusersthefollowingquestion(oranequivalentvariant)abouthowimportanttheVLwastotheirresearch

IfGVLhadnotexisted,wouldyouhavebeenabletoobtaintheresourcesyoulastusedfromanothersource?

ForBCCVL,CVLandGVLtheproportionofuserswhoanswered“No”tothisquestionwas55.26%,29.03%and34.3%respectively.ItisclearthereforethattheVLsareimportantagentsenablingresearchintheirfields.Againbywayofcomparison,about45%ofEBIusersalsoindicatedthattheycouldnothaveproceededintheirresearchwithoutEBI.IfweassumethatthesepercentagestranslateintoresearchenabledbytheVLswhichwouldnototherwiseoccur,thisimpliesthatwecancalculatethevalueoftheresearchundertakenbyVLusersbymultiplyingthenumberofusersbytheirannualaveragecostandapplyingthepercentagesquotedabove.Aswiththeprevioussectionwepresenttworesults.Thefirstusersthecumulativenumberofusersforthecalculationandthesecondusesthenumberofnewusers.Tables6.11and6.12showtheresults.Aswiththevalueofresearchefficiency,thisdemonstratesthattherearesubstantialbenefitsarisingfromtheVLsinenablingresearchwhichwouldotherwisenothaveoccurred,whichevermethodisused.

33

Table6.11 ValueofVirtualLaboratoriesinenablingadditionalresearch,usingcumulativeusers,$

Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 6,520,762 3,781,977 0 6,520,762 3,781,9772014 0 14,651,388 31,427,409 0 7,325,694 15,713,7052015 8,860,592 26,034,265 55,290,864 2,953,531 8,678,088 18,430,2882016 40,814,537 39,043,267 88,712,033 10,203,634 9,760,817 22,178,0082017 104,722,428 53,353,169 137,220,003 20,944,486 10,670,634 27,444,0012018 200,584,264 69,094,061 209,447,547 33,430,711 11,515,677 34,907,9252019 296,446,101 86,409,042 305,661,872 42,349,443 12,344,149 43,665,9822020 392,307,937 105,455,522 405,205,981 49,038,492 13,181,940 50,650,748

Table6.12 ValueofVirtualLaboratoriesinenablingadditionalresearch,usingnewusers,$

Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 4,878,376 2,959,808 0 4,878,376 2,959,8082014 0 8,130,626 27,645,432 0 4,065,313 13,822,7162015 8,860,592 11,382,877 23,863,455 2,953,531 3,794,292 7,954,4852016 31,953,945 13,009,002 33,421,169 7,988,486 3,252,250 8,355,2922017 63,907,891 14,309,902 48,507,970 12,781,578 2,861,980 9,701,5942018 95,861,836 15,740,892 72,227,545 15,976,973 2,623,482 12,037,9242019 95,861,836 17,314,981 96,214,325 13,694,548 2,473,569 13,744,9042020 95,861,836 19,046,480 99,544,109 11,982,730 2,380,810 12,443,014

ReturnstoresearchactivitiesmadepossiblebyVirtualLaboratories

Becausetechnologyisimportantinthecontinuingdevelopmentofnewproductsandimprovedproductivity,theroleofresearchanddevelopmentanditscontributiontotheeconomyhasbeenstudiedwidely.AreviewofthesestudieshasbeenprovidedinSection3abovewhichconcludedthata40%annualreturnoninvestmentisaconservativeestimateofthevalueofresearch.TakingthevalueoftheadditionalresearchduetotheVLsfromTable6.12wecalculatethevalueofthisresearchfollowingtheproceduresetoutbyBeagrieandHoughton.Weassumethattheusefullifeofanyresearchis30yearsandwedepreciatethevalueat3.4%ayearsothatthevalueinyear30iszero.Foreachoftheyears2013to2020asshowninTable6.12,wecalculatetheresultingbenefitsstreambyapplyingthismethodology.Thiswillgenerate8benefitsstreamswhichwesumforeachoftheyears2017to2047.Wethendiscountfuturebenefitstreamsusinga3%discountrateandcalculateanetpresentvalue(NPV)ofthesummedbenefitsstreams.FinallywecanexpresstheNPVofthesebenefitsasanannualised

34

valuebydividingby30.BoththetotalNPVandtheannualisedvalueareshownforeachVLinTable6.13.

Table6.13 ReturnstoadditionalresearchmadepossiblebyVirtualLaboratories$m.

30yearbenefit AnnualisedbenefitBCCVL 2,277.8 75.9CVL 609.5 20.3GVL 2,356.0 78.5

35

7.FurtherassessmentoftheimpactandvalueoftheCharacterisationVirtualLaboratory

DifferentVirtualLaboratories(VLs)sharethesamepurpose,whichistofacilitatedataanalysis,provideresearcherswithITresourcesandpromotecollaborativeresearch.WheretheVLsdifferisintheirscopeandimplementation.AmajorpointofdifferenceforCVLisitsmethodofoperationbecauseitworksprimarilywithinstrumentfacilitiesratherthanendusers.Bydevelopingdatacapture,analysisandvisualisationworkflowsdirectlywithinstrumentfacilities,CVLaimstounderpintheentirefacilityusercommunity.Thisintegrationenablesseamlesstransferofdatafromtheinstrumentsforstorageandanalysis.

TheCVLmethodofoperationandeaseoftransfermeansthatusersarenotalwaysawarethattheyareusingCVL,andmanyconsiderthecapabilitiesprovidedbyCVLasacomponentoftheinstruments,ratherthanasaseparateservice.LowawarenessofCVLamongitsusersraisesconcernsabouttherepresentativenatureofresponsesfromtheCVLusersurvey.

IncollaborationwithDrWojtekGoscinski,themanagerofCVL,thecurrentstudyusedanadditionalapproachtoevaluateCVLfromtheperspectiveofmanagersfromanumberofimagingfacilities.Imagingfacilitieshouseandmaintainadvancedimaginginstruments,andtheyrepresentconsiderablenationalinvestmentinAustralia’sresearchinfrastructure.AlistofimagingfacilitiesthatutiliseCVLservicesisgiveninAppendix2.Todate,CVLhasbeenintegratedwith69instrumentsin26imagingfacilities.

TheinterviewsconductedwithfacilitymanagersprovidedaglimpseintodiverseaspectsofCVLvalue.MuchofthatvalueisadditionaltothedirectbenefitofCVLtoindividualusers.Theadditionalbenefitsincludedeconomiesofscale,thepotentialforscalingupCVLapplicationandtheintrinsicvalueofCVLasadigitalresource.Thesedifferenttypesofvalueareoutlinedbelow.

(i) AcentralisedHPCcapabilitycutscostsbyreducingHPCreplicationacrossresearchorganisations

ImaginginstrumentscangeneratevastamountsofdatathatrequiresextensiveHPCcapabilitiesforanalysis.DrGeorgRamm(CliveandVeraRamaciottiCentreforStructuralCryo-ElectronMicroscopy,MonashUniversity)managesanelectronmicroscopyfacilitythatcontainsmanysuchinstruments,mostnotablytheTitanKrioscryo-electronmicroscope,whichisusedtypicallyfor2Dand3Dcharacterizationofcellularormolecularstructuresanditisparticularlydataintensive.Therearearound100usersatthecentre,ofwhomapproximatelyhalfrelyontheHPCcapabilityofCVLonMASSIVEfordatastorage,whiletheremainderrequiretheanalyticalcapacityofCVLfordataprocessing.

CentralisingHPCatthelevelofeachimagingfacilitywouldbeacheaperalternativetouser-fundedHPC,thoughstillconsiderablymoreexpensivethanthecostofmaintainingCVL:

36

• Asmallcomputingclusterwouldcostaround$200,000anditwouldneedtobeupdatedwithin5years.Salarycostsforpart-timePhD-levelstafftomanagetheclusterwouldaddaround$50,000peryeartotheoutlay,bringingthetotaltoaround$90,000peryear.

• DrAndrewJanke(CentreforAdvancedImaging,UniversityofQueensland)estimatedCVLreplacementcostat$250,000.Theamountwasbasedonthecostofsettingupanin-houseplatformforthecentre.

• ProfSampson(CentreforMicroscopy,CharacterisationandAnalysis,UWA)estimatedthatwhenimplementedatfullcapacity,thevalueofCVLservicestothecentrecouldwellbeworth$50,000peryear.Atthisstage,flowandmasscytometeristheonlyinstrumentintegratedwithCVL,althoughintegrationwithotherinstrumentsisbeingrolledout.

• DrIanHarperfromMonashMicroImaging(MMI)estimatedthecostofreplacingtheaccesstoHPCthroughCVLat$20,000peryear.

UsingDrHarper’smostconservativeestimate,itwouldcostover$500,000peryeartoreplaceHPCservicescurrentlyprovidedbyCVLto26facilities.ThisfigureiswellabovetheCVLrunningcostof$177,000-$320,000peryear(Table2.2).TheestimatedreplacementcostisforHPCalone;itdoesnottakeintoaccountthevalueofallotherservicesprovidedbyCVL,whichareoutlinedbelow.

(ii) PotentialforscalinguptheCVLplatform

CVLhasbeenadoptedwidelyinarelativelyshorttime.Inlessthanfouryearssinceitsinception,ithasbeenintegratedwith69instrumentsacross26imagingfacilities.TheexpansionofCVLisongoing.Inaddition,CVLintegrationwithcommonlyusedtechnologyhasthepotentialforscalingup.Forexample,theintegrationoftheflowcytometerattheCentreforMicroscopy,CharacterisationandAnalysisinUWA,isexpectedtobereplicatedwidely.Flowcytometryisincreasinglyusedforcellcountingandsortingnotonlyinresearch,butalsoinmajorhospitals.Thereareover20machinesandaround100usersofflowcytometryinWA,over100machinesinAustraliaandaround5,000worldwide.IntegrationwithCVLhasthepotentialtobereplicatedinothercentres,attractingmoreuserstotheplatform.

Lightmicroscopyisanotherwellestablishedandcommontechnology.DrIanHarperfromMonashMicroImaging(MMI),outlinedplanstointegrateMMIinstrumentswithCVL.MMIisalightmicroscopyfacilitythatcontainsinstrumentsvaluedataround$50million,andwhichattractapproximately500users.MMIislocatedonthreecampuses:MonashUniversity(Clayton),AlfredHospitalandMonashMedicalCentre.AtthisstageonlytheinstrumentsattheClaytoncampusareintegratedwithCVL.Withinthecomingyear,40-48MMImicroscopesatallthreecampusesareexpectedtobeintegratedwithCVL.

37

(iii) Increasingtheefficiencyofresearchandtheimpactofresearchinvestment

Inprovidingaccesstosecurestorageforexperimentaldata,CVLcontributestoimprovedresearchoutcomes.DrIanHarperfromMonashMicroImaging(MMI)describedproblemswithdatalosspriortointegrationwithCVL,whenUSBdatatransferwasused.DrRamm(CentreforStructuralCryo-ElectronMicroscopy)alsorecalledprolongedperiodsofdowntimewhenusersbroughttheirownharddrivestodownloaddata,makinginstrumentsvulnerabletoattackbycomputerviruses.

Theabilitytostoreallexperimentaldatasecurelyallowsresearcherstointerrogateolddatainsubsequentyearstoanswernewresearchquestions.TheabilitytosharedatawithcollaboratorsthroughtheCVLplatformalsomeansthatthesamedatacanbeinterrogatedbyotherresearchers.Reusingexperimentaldatainthiswayandimprovestheefficiencyandimpactofeachexperiment;italsoreducestheneedtorepeatexperiments

Increasingreproducibilityofresearchfindingsisanotherwaytoimprovetheefficiencyandimpactofexperimentalresearch.DrAndrewJanke(CentreforAdvancedImaging,UQ)emphasisedthatoneofthemainbenefitsofCVLtohisfacilityisthatitfacilitatesreproducibility.DrJankeusedtheexampleof“quarantining”supersededversionsofanalyticalsoftware,suchasFSL,apackageofimageanalysistoolsusedforMRIbrainimagingdata.AfterareleaseofupdatedFSLolderversionsarenolongeravailable.Thisoftenmeansthatresultsofearlierexperimentscannotbereplicated.Arecentstudyestimatedthatfewerthan16%ofneuroimagingresultsarereproducible(Russelletal2017).CVLpreservesallreleasesofthesoftware.

DrCeguerra(AustralianCentreforMicroscopy&Microanalysis,UniversityofSydney)notedthatintheabsenceofCVL,thereturnonthe$5millioninvestmentintheAtomProbewouldbegreatlydiminishedintermsofresearchoutcomes.TheAtomProbeenables3Dimagingandchemicalcompositionmeasurement.Becauseofthenoveltyofthistechnology,therearefewanalyticaltoolsavailableandusersarerequiredtodeveloptheirownmethodsandsoftware.TheAtomProbeworkbenchdevelopedbyCVLactsasarepositoryofanalyticalsoftwarethatisdevelopedbyusers.Thisrepositoryrepresentsasignificantinvestmentinresearchtime–over14yearsofeffortfrommultiplepeople.AccordingtoDrCeguerra,withoutCVLsomeprojectswouldbesetbackbyasmuchasfiveyears,withtheresearchershavingtodevelopthetoolstheyneedbythemselves.CVLisalsocriticalfordisseminatingresearchoutcomesintheAtomProberesearchcommunity.

(iv) ValueofCVLasadigitalresource

AnotherimportantaspectofthevalueofCVListhedevelopmentofitsintrinsicworthasadigitalresourceis.DrCeguerraemphasisedtheimportanceofCVLasaresourceforthesmallcommunityofcommunityofresearchersworkingwiththeAtomProbe.Thereare

38

closetoonehundredsuchinstrumentsworldwide2witharound500-600researcherstrainedtousethem,34ofthematthefacilityinSydney.Despiteitssmallsize,thiscommunityrepresentanimportantfieldofresearchthathasproducedfar-reachingdiscoveriesinmaterialsengineering.

Asalreadymentioned,theatomprobeisnoveltechnologywithoutacanonofwell-establishedanalyticalmethods.Ithasfallentouserstodevelopthemethodstheyneed.Therepositoryofuser-generatedsoftwareontheCVLAtomProbeworkbenchhasnoequivalentandhasbecomethekeyplatformforAtomProberesearchworldwide.CVLisalsocriticalfordisseminatingresearchoutcomesintheAtomProberesearchcommunity.ThevaluationoftheintrinsicvalueofVLsasadigitalresourceisoutlinedinmoredetailinSection8ofthisreport.

DrAndreJankesaidthatCVLprovidednoviceuserswithaneasytonavigatemechanismfortorapidlygetuptospeedwiththeiranalysiswithouttheneedtoinstallorconfiguresoftware,somethingthatpreviouslytookalargeinvestmentoftime.DrJankeestimatedthesavingintimeasapproximatelythreemonths.Thattimeisvaluedatover$19,000juniorforapostdoctoralfellow(commencementlevelAcademiclevelAsalaryis$77,819,Table6.1)or$6,670foraPhDstudent(MonashUniversityPhDannualstipendsare$26,682)

Interviewswithimagingfacilitiesmanagersprovidedaninsightintofar-reachingbenefitsofCVL,whichwouldnotbeapparentfromsurveyresponsesofindividualusers.ThecategoriesofvalueidentifiedinthiswaywouldbeequallyapplicabletotheotherVLs.Inasimilarway,BCCVL,GVLandHuNIoffereconomiesofscalethroughcentralisingITresources;andaimtodevelopvalueasdigitalresources,toimprovetheefficiencyandimpactofresearchandtoexpandtheirservicesoutsidethehostorganisationsandoutside

2AtomProbefacilitiesaroundtheworld(http://www.atomprobe.com/2ndLinks/apt-internet-sites.aspx

39

8.QualitativevaluationofNectarVirtualLaboratories

TheimpactandvalueofNectarVLscalculatedintheSection6arenecessarilyrestrictedtothoseforwhichthekeyinputstothecalculationcanbereadilyquantifiedandexpressedineconomicterms.

ItisclearhoweverthattheactivitiesofVLsextendwellbeyondthisintoareaswhicharemoredifficulttoquantify.WeattempttocapturesomeofthequalitativebenefitsofNectarVLsusingtheBalancedValueImpactModelframeworksuggestedbyTanner(2012).Table8.1providesabriefdescriptionofthesetypesofvalue.Table8.2attheendofthissectionsummarisesimportantaspectsofthevaluesdescribedinTable8.1forBCCVL,CVLandGVL.

Table8.1 TypesofvalueinBalancedValueImpactframework

TypeofValue Definition

UtilityValue Peoplevaluetheutilityaffordedthroughuseofthedigitalresourcesnoworsometimeinthefuture.

Existenceand/orPrestigeValue

Peoplederivevalueandbenefitfromknowingthatadigitalresourceischerishedbypersonslivinginsideandoutsidetheircommunity.Thisvalueexistswhethertheresourceispersonallyusedornot.

EducationValue Peopleareawarethatdigitalresourcescontributetotheirownortootherpeople’ssenseofculture,education,knowledgeandheritageandthereforevalueit.

CommunityValue Peoplebenefitfromtheexperienceofbeingpartofacommunitythatisaffordedbythedigitalresource.

Inheritance/BequestValue Peoplederivebenefitfromtheinheritancepasseddowntothemandsatisfactionfromthefactthattheirdescendantsandothermembersofthecommunitywillinthefuturebeabletoenjoyadigitalresourceiftheysochoose.

BasedonTanner(2012)p37

Utilityvalue

TheprecedingsectionhasshownthevaluethatusersputonVLservicestotheextentthatthiscanbeexpressedineconomicterms.

40

Intheon-linesurveysuserswereaskedthefollowingquestion

TowhatextentdoyoubenefitfromusingVLinanyofthefollowingways?

Haven'tused,Nobenefit,Lowbenefit,Mediumbenefit,Highbenefit,Veryhighbenefit

Table8.2 ExtentofbenefitsfromVirtualLaboratories,%ofrespondents

Services BCCVL CVL GVLMediumandhigher

Highandveryhigh

Mediumandhigher

Highandveryhigh

Mediumandhigher

Highandveryhigh

Data&tools 73.8 57.4 72.5 70.0 60.6 44.8

Collaborations 23.0 11.5 50.0 30.0 48.7 25.0

Training 37.7 27.9 27.5 15.0 51.2 32.9

Usersupport 27.9 19.7 --- --- 36.8 29.0Other 27.9 19.7 7.5% 2.5% 36.8 29.0

Intheon-linesurveysuserswereaskedthefollowingquestion

WhatimpactwouldithaveonyourworkorstudyifyoucouldnotaccessVLservicesandresources?

Noimpact,Slightimpact,Moderateimpact,Majorimpact,Severeimpact

ForBCCVLusers18.2%answeredthatiftheycouldnotaccesstheservicesandresourcesprovidedbytheVLthiswouldhaveamajororsevereimpact.ForCVLandGVLthepercentageswerehigherat52.8%and35.7%respectively(Table8.3).

Howevertakingmoderate,majorandsevereimpactassignificant,thepercentagesofrespondentswere41.8%,75.0%and72.9%.

Table8.3 ImpactifcouldnotaccessVirtualLaboratoryservices,%

BCCVL CVL GVLNoimpact 23.6 5.6 11.4Slightimpact 34.5 19.4 15.7Moderateimpact 23.6 22.2 37.1Majorimpact 12.7 27.8 27.1Severeimpact 5.5 25.0 8.6 Majorandsevereimpact 18.2 52.8 35.7Moderate,majorandsevereimpact 41.8 75.0 72.9

41

CVL

WeconductedphoneinterviewswiththemanagersofanumberofimagingfacilitiesthathaveworkedcloselywithCVLonintegratingtheirfacilitieswiththeon-linestorageandcomputingfacilitiesofferedthroughCVL.

TheyprovidedthefollowingdescriptionsofthevalueofCVLtotheirfacilities.

DrKeithSchulze,ImageAnalyst,MonashMicroImaging,MonashUniversity

MyDataandStore.MonashrepresentasignificantadvanceindatahandlingatMonashMicroImaging.Transferandstorageofuserdataisnowreliable,seamlessandsecure.Store.Monashprovidesuserswiththeabilitytosharedatawithcolleaguesandcollaboratorsinaconvenientandsecuremanner.Italsohasmechanismstocaptureandexposeimportantmetadatathatallowforbetterreuseandreproducibilityofdata.

Next-generationimagingtechnologies,liketheLatticelight-sheetmicroscope,arecapableofproducingdataatarateofTerabytesperhour.ThetoolsforautomatedhandlingandstorageofdataprovidedbyCVLareakeyenablerforresearcherstoderivethemostbenefitfromtheselargedatasetsi.e.,theyspendlesstimestrugglingwithdatatransfersandmoretimeextractinginterestinginformationfromtheirdata.Moreover,CVLandNectarprovideacrucialplatformonwhichtoolstoanalyseandvisualthisdatacanbedevelopedanddeployed.

DrAndrewMehnert,GroupLeader–DataManagement,AnalysisandVisualisation,CentreforMicroscopyCharacterisationandAnalysis,TheUniversityofWesternAustralia

TheCentreforMicroscopy,CharacterisationandAnalysis(CMCA)isaUniversityfacilitythatcollaboratesinmicroscopyandcharacterisation,supportingresearchexcellencelocally,nationally,andinternationally.Itcomprises~40staff,morethan500usersandhosts~50instrumentplatformsacrosscytometry,opticalmicroscopy,micro-magneticresonanceimaging(microMRI),preclinicalbioimaging,electronmicroscopy(EM),X-raymicro-computedtomography(microCT),secondaryionmassspectrometry,bio-organicmassspectrometry,X-raydiffractionandnuclearmagneticresonance(NMR)spectroscopy.TheCMCAcollaboratesinandsupportsresearchacrossbiologicalscience,biomedicalscience,earthscienceandphysicalscience.

CMCAusersareacquiringever-largermulti-dimensionaldatasetsandareincreasinglyusingmultipleinstrumentplatforms;e.g.microCTtogetherwithEMandRamanspectroscopy.ThispresentsaBigDatachallenge,notonlyintermsofmanaging/curatingthisdataovertheresearchlifecycle,butalsointermsof

42

analysingthisdatatofacilitatenewdiscoveries.TosolvethischallengetheCMCAisleveragingtheCharacterisationVirtualLaboratory(CVL)andCVL-supportedservicesincludingMyTardis.Overthelast12monthstheCMCAhasbeenworkingwithMonashUniversitytodeveloptheCytometryworkbench(collectionofsoftwaretools)fortheCVLtosupporttheanalysisofdatafromimage-,mass-andflow-cytometryinstruments.TheCMCAhasalsointegratedseveralEMinstrumentswithMyTardisandisintheprocessofintegratingitsNIFflagshipMRIscanner.TheCMCAplanstointegrateallofitsinstrumentswithMyTardisandtodevelopadditionalworkbenchesandworkflowsintheCVLinsupportofitsusers/researchers.TheCVLandsupportedservicesrepresenttheonlyviablesolutionacrossmultipleinstrumentplatformsandmodalities.Moreover,lookingtothefuture,theCVLoffersasolutiontotheanalysisandvisualisationofverylargedatasetsusingtoolsandcomputeresourcesnototherwiseavailablefromadesktopworkstation.

DrAndrewJanke,InformaticsFellow,NationalImagingFacility,UniversityofQueensland

ThevalueofCVLandmanagedworkbenchesinneuroimagingandpreclinicalimagingtotheNationalImagingFacility(NIF)ispredominatelyaroundencouragingouruserstoperformreproduciblescience.CVLprovidesatoolinwhichresearcherscanprocessandanalysetheirimagingdatasafeintheknowledgethatiftheyneedtoreproducetheanalysisin5years’timethesameversionsofsoftwarewillstillbeavailable.

Traditionallythishasnotbeenthecaseinimagingresearchandsoftwarepackagesandoperatingsystemsareamovingtarget.CVLprovidesamechanismtofreezeintimeaparticularanalysispipeline.InadditionCVLprovidesNIFwithaneasytonavigatemechanismfornoviceuserstorapidlygetuptospeedwiththeiranalysiswithouttheneedtoinstallorconfiguresoftware,somethingthatpreviouslytookalargeinvestmentofNIFpersonneltime.

Educationvalue

Intheon-linesurveysBCCVLandGVLuserswereaskedthefollowingquestion

HowdidyoulearntousetheservicesprovidedbytheVL?

Asshowninthetablebelow,53.8%ofBCCVLusers,50.0%ofGVLusersrelyontherespectiveVLstolearntheanalyticaltoolsrequiredfordataanalysisintheirrespectivefields.

AsisevidentfromTable8.4,animportantaspectofbothBCCVLandGVListhetrainingtheyprovidetoresearchersandstudents.

43

Table8.4 TrainingprovidedbyVirtualLaboratories,%ofrespondents

BCCVL GVLAlreadyproficient 11.5 21.1TutorialthroughtheVLwebsite 21.3 18.4Face-to-facecourseprovidedbyVLpersonnel 32.8 9.2Face-to-facecourseprovidedbysomeoneelse 0.0 22.4Anotherway 16.4 15.8Missing 18.0 13.2Total 100.0 100.0

OneoftheimportantbenefitsofGVListoprovidetouserstheresourcestheyneedwhichotherwisewouldrequireadvicefromexperiencedbio-informaticians.Thesupplyoftrainedbio-informaticiansisacriticalbottleneckinexpandingresearchwithinthefieldofgenomics,soenablingresearcherstobypassthisbottleneckisanimportantcontributionthatGVLmakes.

OvertheperiodtoFebruary2017,GVLhasundertakentrainingcoursesfor313users,whogiveanaverageratingof4.5outof5forthesecourses.

Benefitstousers

BCCVL

InitsSnapshotfor2016,BCCVL(2016)providesthefollowingquotesfromusers.

...becauseitprovidesanenvironmentforexperimentingwithmodels,andalsokeepscarefultrackoftheseexperiments,BCCVLalsoencouragesbestpracticeandtransparencyinmodelling.

Withitseasytouseinterface,accessiblefromanywhere,itopensthefieldtoawholenewarrayofresearcherswhounderstandthesystemstheyareworkingon,butdonothavethetechnicalskill-setsorhardwaretoproperlyanswertheirquestions.

…essentiallytheBCCVLhasenabledustoaskquestionsthatwecouldn’taskbefore–questionswemayhavewantedtoaskbutcouldn’tlogisticallyhopetoanswer,soit’sopenedupawholenewfieldofenquiry.

CVL

Initsannualreportfor2015-16,theMassiveprojectatMonashUniversitynotedthat

CVLhasunderpinnedworkflowdeploymentattwoAustralianSynchrotronbeamlinesforaccesstotheMASSIVEDesktop.ResearchersvisitingboththeImagingandMedicalBeamline(IMBL)andtheX-rayFluorescenceMicroscopy(XFM)beamlineareautomaticallycreatedabeamlinevisitproject,anduseraccounts.Authenticationis

44

integratedwithAustralianSynchrotronsothatuserscanusetheirAScredentialstologintothesystem.Asetofbeamlinechangeoverscripts,developedincollaborationwithAustralianSynchrotron,controltheflowofdata.Theimpactofthisworkissignificant.Researchersareprovidedwithaccesstoaremotedesktopenvironmentforthedurationoftheirbeamlinevisitthatprovidesaccesstothedatacaptured,andtoolsfordataprocessingandvisualisation

Additionally,CVLsoftwareisnowbeingadoptedbytwomajorinternationalsupercomputingcentres:

>JulichSupercomputingCentre,isintheprocessofdeployingStrudeltosupportvisualisationusersandhigh-endcommercialengineeringapplications;

>EdinburghParallelComputingCentreforindustryaccesstoHPCsystemsundertheEUFP7project,Fortissimo.

WithCVLNectarfunding,MonashUniversityisdevelopinganinstrumentintegrationapp,calledMyDatatomakeintegrationquicker,simplerandlessreliantonspecialistITsupport.AsaresultofthisprojectMyDataisnowusedat35instruments,across10facilities,at4institutions,inaddition17furtherinstrumentsareplannedat4institutions.Inaddition,thisprojecthasmadeMyTardiseasiertouseforFacilityManagers.(p23)

HuNI

DescriptionsbytwoHuNIusersofitsvalue,whichincludesnewapproachestomulti-disciplinaryresearchandconnectingwithotherresearchers.‘Userscancapturerelationshipsbetweencontent,buildpathwaysandstructures,andshareanddistributedata.Itprovidesaplatformwhereserendipitouscollaborativeopportunitiescanquicklyemerge’(User2).

User1(literarystudies)

Iworkintheareaofliterarystudies,particularlyliteraryandculturalhistoryinthenineteenthcentury,andmyresearchcutsacrossarangeaphysicalarchivesandlibrariesaroundtheworld.WhileIuseanumberofdigitaltoolsinmyresearch(particularlyasdigitisationhasopenedupaswatheofhistoricprintedtexts),HuNIopenedupmythinkingandapproachestoresearchinwaysthattheseindividualtoolshadneverencouraged.Inparticular,HuNIinnatelyencouragesuserstoidentifytheinterdisciplinaryconnectionswithintheirresearchfield.Asourresearchmethodshavelargelybeenshapedwithinourdisciplinarysilos,IwasnotevenawareofsomeofthedatasetsthatHuNIdrawsonbutwhichprovedtobesupremelyusefultomyresearchproject.Moreover,HuNIrepresentsarealstepforwardinefficiencyinthatsimpleawarenessoftheseotherdatasetsontheirownwouldstillmeanincreasedlabourin

45

individuallylocatingandsearchingthem.PerhapsmostexcitingandunexpectedofallwastheabilityofHuNItoconnectmyideasandresearchwiththeworkofotherresearchersthroughitsvisualisationcapabilities.Thesekindsofassociationsinrelationtospecificresearchsites(particularplays,books,orpublicfigures,forexample)aresimplynotpossibleinanyotherforumapartfromtherandomnetworksweformasacademics.Inthisrespect,HuNIclearlyhasthepotentialtoradicallytransformhowweconductourresearchinrelationto—andpotentiallyintandemwith—otherscholarsandmembersofthepublic.

User2(Archivist)

HuNI(HumanitiesNetworkedInfrastructure)makesasignificantandvaluablecontributiontoAustralia’sinformationinfrastructure.Inbringingtogether30curated,authoritativeandscholarlydatasetsinawell-designedonlineresource,HuNIsupportsdiverseresearchandprovidesresearcherswithdiversecontentwithoutthenoiseoflargeaggregators.Throughthis,HuNIalsohelpsincreasethevisibilityanduseofthese30foundationalhumanitiesresourceswhichtogetherconstituteasubstantialinvestmentofresourcesandscholarshipspanningmorethantwodecades.

Moreimportantly,unlikeaggregatorsand‘portals’,HuNIbringsthiscontentintoaresearchplatformwhichsupportsthetypeofrelational,non-hierarchical,iterativeengagementwithcollectionsanddatathatiscentraltocontemporaryhumanitiespractice.Userscancapturerelationshipsbetweencontent,buildpathwaysandstructures,andshareanddistributedata.Throughthis,HuNInotonlyfostersserendipitousdiscovery,itprovidesaplatformwhereserendipitouscollaborativeopportunitiescanquicklyemergeinanonlineenvironment.Astheuserbasegrows,thisaspectisincreasinglylikelytosparkbroadercollaborativeworkbyhumanitiesresearchers,contributingtothestrengthanddiversityofthesectorasawhole.

46

Table8.5 SummaryofbenefitsprovidedbyBCCVL,CVLandGVLusingBalancedValueImpactframework

Valuedriversandindicators BCCVL CVL GVL

1.UtilityValue• Reportedbenefitfromaccesstodataandtools1

ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

45(73.8%)35(57.4%)

29(72.5%)28(70.0%)

46(60.6%)34(44.8%)

2.Existenceand/orPrestigeValue• reachacrossdifferentorganisations[e-mailsuffixes]2

NumberofUniversitiesNumberofResearch/Not-for-profitorganisationsNumberofGovernmentagencies/departmentsNumberofCompanies

• ReachoutsideAustralia2Numberofcountries(includingAustralia)Australia(numberofusers)Othercountries(numberofusers)

• recognitionbyprovidersofotherdigitalplatforms

70(46.7%)20(13.3%)38(25.3%)22(14.7%)201005(94.8%)55(5.2%)Seetext

46(49.5%)33(35.5%)7(7.5%)7(7.5%)152,388(93.4%)170(6.6%)Seetext

24(64.9%)8(21.6%)--4(10.8%)1665NoneSeetext

3.EducationValue• trainingtroughcoursesbyVLpersonnel/VLwebsite

• VLtraining

ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

• satisfactionsurveysforGVLcourses(meanscoreoutof5)

33(54.1%)23(37.7%)17(27.9%)--

--

11(27.5%)6(15.0%)--

21(27.6%)39(51.2%)25(32.9%)4.5

4.CommunityValue• useofVLplatformsforcollaboration

ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

• usersupportofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

14(23.0%)7(11.5%)17(27.9%)12(19.7%)

20(50.0%)12(30.0%)

----

37(48.7%)19(25.0%)28(36.8%)22(29.0%)

5.Inheritance/BequestValue• VLusersuploadnewdatasetstoaVL(numberofrespondents)• NumberofdatasetsuploadedbyDec2016• Repositoryofuser-writtenalgorithms

14(22.9%)

----

--

35,787yes5

22(29.0%)

--yes

1SurveyofVLusers2Informationderivedfromusere-mailsuffixes3Studentfeedbackfromparticipantsin31GVLtrainingcourse(informationprovidedbyVL

47

9.Summaryandconclusions

TheNectarfundingforthreeVLsexaminedinthisstudyhashaddemonstrablebenefits,whenmeasuredinbothquantitativeandqualitativeterms.ForanexpectedexpenditurebyNectarandpartnersofaround$4.4to$7.0milliontotheyear2020(oraround$550,000to$870,000inannualterms)thethreeVLshavegeneratedeconomicvalueexceedingthis.

Table9.1summarisesthecalculationofeconomicbenefitsexpressedinannualtermsandcomparesthistotheannualisedcostofeachVL.

Table9.1 SummaryofeconomicbenefitsandcostsforVirtualLaboratories

Annualisedbenefit,$

Annualisedcost,$

Ratio

Contingentvaluation Willingnesstopay

BCCVL 2,324,397 550,754 4.2CVL 1,235,410 612,356 2.0GVL 1,901,364 869,479 2.2

Willingnesstoaccept BCCVL 20,142,090 550,754 36.6CVL 11,454,271 612,356 18.7GVL 4,031,769 869,479 4.6

Efficiencyimpacts BCCVL 4,087,052 550,754 7.4CVL 1,938,437 612,356 3.2GVL 14,184,310 869,479 16.3

Additionalresearchimpact BCCVL 11,982,730 550,754 21.8CVL 2,380,810 612,356 3.9GVL 12,443,014 869,479 14.3

Returnstoadditionalresearch BCCVL 75,925,851 550,754 137.9CVL 20,318,154 612,356 33.2GVL 78,532,855 869,479 90.3

TheratiosforallVLsarehigherthanoneindicatingthattheyallgeneratebenefitsinexcessoftheircosts.ForBCCVLandGVLtheratiosareconsistentlyhighacrossallmeasuresofvalueandarehigherthanthoseoftenachievedbyconventionalphysicalinfrastructure.ThestudyindicatesthattheVLscontributevalueindifferentways.Forinstance,theefficiencyimpactratioforGVLishigherthanthatforBCCVL,thecontingentvaluationratioandtheadditionalresearchratioarehigherforBCCVLthanforGVL.ThesevariationsreflectdifferencesinthewayinwhichtheVLsdelivervaluetotheirusers.

48

Takingawiderperspectiveonbothquantitativeandqualitativebenefits,itisclearfromthecasestudyofCVLthatithasgeneratedconsiderablebenefitsfortheimagingfacilitieswithwhichitworks.Thesefacilitieswouldotherwiseincurconsiderablecosts(discussedinSection7butnotfullycapturedbythestudy)inreplicatingtheservicestheyprovide.ThefacilitymanagershavealsoprovidedevidenceofthewiderbenefitsofworkingwithCVL.WehaveusedtheBalancedValueImpactframeworktodescribecharacteristicsofVLsthatprovidedarangeofbenefitstousersandcommunities.ThisapproachcouldbeadaptedfurtherforprovidinginsightintothevalueandimpactofotherVLs.

WenotedintheIntroductionthatmostVLsfundedbyNectarhaveonlybeenactiveforafewyearsandarestillintheirgrowthstages.AnevaluationoftheiroverallimpactandvaluemightbestbedonefromtheperspectiveofsomeyearsinthefuturewhentheVLsareinamorematuregrowthphase.Thereforetheanalysisandconclusionsdrawninthisstudyshouldbetreatedaspreliminaryanddependsignificantlyontheassumptionsmadeaboutfuturegrowthpaths.

AlthoughVLssharemanyfeaturesincommon,theydiffersignificantlyfromeachotherintermsoftheservicestheyprovidedtotheirtargetcommunities.ThismeansthattheapproachtotheevaluationofvalueandimpactmustbetailoredtothecircumstancesofeachoneandthatconsiderableeffortshouldbemadeinunderstandingtheservicesprovidedbyeachVLandhowtheseservicesaredeliveredtoandbenefittheirtargetcommunities.

49

References

ABS2017,6302.0AverageWeeklyEarnings,Australia,athttp://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/6302.0Nov%202016?OpenDocument

AccessEconomics2008,Exceptionalreturns:thevalueofinvestinginhealthR&DinAustraliaII.Canberra(Australia):AustralianSocietyforMedicalResearch;2008.

AfganE,SloggettC,GoonasekeraN,MakuninI,BensonD,CroweM,GladmanS,KowsarY,PheasantM,HorstR,LonieA.,GenomicsVirtualLaboratory:APracticalBioinformaticsWorkbenchfortheCloud.,PLoSOne.2015Oct26;10(10):e0140829.

BarkerMichelle,NancyWilkins-Diehr,SandraGesing,DanielS.Katz,SilviaDelgadoOlabarriaga,ShayanShahand,ScottHenwood,TristanGlatard,KeithJeffery,BrianCorrie,AndrewTreloar,HelenGlaves,LesleyWyborn,NeilChueHong,AlessandroCosta2017,TheGlobalImpactofScienceGateways,VirtualResearchEnvironmentsandVLs,

BarkerM2016,TheAustraliansciencegatewaysecosystem.PeerJPreprints;2016Oct6.

BCCVL2016,Our2016Snapshot,Jan5,2017,availableathttp://www.bccvl.org.au/our-2016-snapshot/

Beagrie,N.,Lavoie,B.,andWoollard,M.(2010)KeepingResearchDataSafe2FinalReportLondon:Jisc.http://www.jisc.ac.uk/publications/reports/2010/keepingresearchdatasafe2.aspx#downloads.

Beagrie,N.,Houghton,J.W.,Palaiologk,A.,andWilliams,P.(2012)EconomicImpactEvaluationoftheEconomicandSocialDataService(ESDS),EconomicandSocialResearchCouncil,London.http://www.esrc.ac.uk/files/research/evaluation-and-impact/economic-impact-evaluation-of-theeconomic-and-social-data-service/

Beagrie,N.andHoughton,J.W.(2013a)TheValueandImpactoftheArchaeologyDataServices:AStudyandMethodsforEnhancingSustainability,JointInformationSystemsCommittee,BristolandLondon.http://www.jisc.ac.uk/whatwedo/programmes/preservation/ADSImpact.aspx

Beagrie,N.andHoughton,J.W.(2013b)TheValueandImpactoftheBritishAtmosphericDataCentre,JointInformationSystemsCommitteeandtheNaturalEnvironmentResearchCouncilUK,BristolandLondon.http://www.jisc.ac.uk/whatwedo/programmes/di_directions/strategicdirections/badc.aspx

50

Beagrie,N.andHoughton,J.W.(2014)TheValueandImpactofDataSharingandCuration:ASynthesisofThreeRecentStudiesofUKResearchDataCentres,JointInformationSystemsCommittee(Jisc),BristolandLondon.http://repository.jisc.ac.uk/5568/DepartmentofInnovation,Industry,ScienceandResearch2008,StrategicRoadmapforAustralianResearchInfrastructure,August2008,availableathttps://docs.education.gov.au/system/files/doc/other/national_collaborative_research_infrastructure_strategic_roadmap_2008.pdf

Beagrie,NeilandHoughton,John2016.TheValueandImpactoftheEuropeanBioinformaticsInstitute,Fullreport,January2016,EMBL-EBI

BritishLibrary.(2004)MeasuringtheEconomicImpactoftheBritishLibrary,Powerpointpresentationonunpublishedreport.http://www.bl.uk/aboutus/stratpolprog/increasingvalue/publicvalue/confpres/pungelwesmarks.pdf

CeguerraAV,LiddicoatPV,RingerSP,GoscinskiWJ,AndroulakisS.Atoolforscientificprovenanceofdataandsoftware.InComputationalScienceandEngineering(CSE),2013IEEE16thInternationalConferenceon2013Dec3(pp.561-565).IEEE.

CommonwealthofAustralia,HandbookofCostBenefitAnalysis,January2006,FINANCIALMANAGEMENTREFERENCEMATERIALNO.6athttps://www.finance.gov.au/sites/default/files/Handbook_of_CB_analysis.pdf

DeloitteAccessEconomics2014,Extrapolatedreturnsfrominvestmentinmedicalresearchfuturefund(MRFF)AustralianSocietyforMedicalResearch17October2014

DeloitteAccessEconomics2015,TheeconomiccontributionsofAustralia’sresearchuniversities–theUNSWexampleFinalreportUNSWAustraliaOctober2015

DepartmentofEducation2017,NationalCollaborativeResearchInfrastructureStrategy(NCRIS),availableathttps://www.education.gov.au/national-collaborative-research-infrastructure-strategy-ncris

EvaRIO2013b,Corestudy:AdaptingtheBETAmethodtothecaseoftheevaluationoftheimpactofResearchInfrastructures,Finalreport:ResultsandPolicyRecommendations-PARTI.

Goecks,J.;Nekrutenko,A.;Taylor,J.;GalaxyTeam,T.(2010)."Galaxy:Acomprehensiveapproachforsupportingaccessible,reproducible,andtransparentcomputationalresearchinthelifesciences".GenomeBiology.11(8):R86.doi:10.1186/gb-2010-11-8-r86.PMC2945788Freelyaccessible.PMID20738864.

Goscinski,W.J.,P.McIntosh,U.C.Felzmann,A.Maksimenko,C.J.Hall,T.Gureyev,D.Thompson,A.Janke,G.Galloway,N.E.B.Killeen,P.Raniga,O.Kaluza,A.Ng,G.Poudel,

51

D.Barnes,T.Nguyen,P.BonningtonandG.F.Egan(2014)."TheMulti-modalAustralianScienceSImagingandVisualisationEnvironment(MASSIVE)highperformancecomputinginfrastructure:applicationsinneuroscienceandneuroinformaticsresearch."FrontiersinNeuroinformatics8.

GoscinskiWJ,BamberyK,FelzmannC,HallC,HinesC,MaksimenkoA,McIntoshP,PanjikarS,PatersonD,RyanC,ThompsonD.MASSIVE:anHPCCollaborationtoUnderpinSynchrotronScience.In15thInt.Conf.onAcceleratorandLargeExperimentalPhysicsControlSystems(ICALEPCS'15),Melbourne,Australia,17-23October20152015Dec(pp.640-644).JACOW,Geneva,Switzerland.

GraduateCareersAustraliaLtd,GraduateSalariesReport2015,GraduateCareersAustraliaLtd,2016athttp://www.graduatecareers.com.au/research/researchreports/graduatesalaries/

HallgrenW,BeaumontL,BownessA,ChambersL,GrahamE,HolewaH,LaffanS,MackeyB,NixH,PriceJandVanderwalJ.2016.TheBiodiversityandClimateChangeVirtualLaboratory:Whereecologymeetsbigdata.EnvironmentalModelling&Software,76,pp.182-186.

Houghton,J.W.andSheehan,P.(2009)‘Estimatingthepotentialimpactsofopenaccesstoresearchfindings,’EconomicAnalysisandPolicy39(1).http://www.eap-journal.com/vol_39_iss_1.php

Houghton,J.W.,Rasmussen,B.,Sheehan,P.J.,Oppenheim,C.,Morris,A.,Creaser,C.,Greenwood,H.,Summers,M.,andGourlay,A.(2009)EconomicImplicationsofAlternativeScholarlyPublishingModels:ExploringtheCostsandBenefits,ReporttoTheJointInformationSystemsCommittee(JISC).http://www.jisc.ac.uk/publications/reports/2009/economicpublishingmodelsfinalreport.aspx

IndustryCommission(1995)ResearchandDevelopment,ReportNo44,15May1995,AustralianGovernmentPublishingService,Canberra.

Massive.2016,MassiveAnnualreport2015-2016,availableathttps://www.massive.org.au/images/stories/MASSIVE_AR_15-16_small.pdf

PoldrackRussellA.,ChrisI.Baker,JokeDurnez,KrzysztofJ.Gorgolewski,PaulM.Matthews,MarcusR.Munafò,ThomasE.Nichols,Jean-BaptistePoline,EdwardVul&TalYarkoni,Scanningthehorizon:towardstransparentandreproducibleneuroimagingresearch,NatureReviewsNeuroscience18,115–126(2017)doi:10.1038/nrn.2016.167.

Shahand,S.Sciencegatewaysforbiomedicalbigdataanalysis[dissertation].Amsterdam:UniversityofAmsterdam;2015.178p.http://hdl.handle.net/11245/1.490613.

52

Shanks,S.andZheng,S.2006,EconometricModellingofR&DandAustralia’sProductivity,ProductivityCommissionStaffWorkingPaper,Canberra,April.

Tanner,S.,2012.MeasuringtheImpactofDigitalResources:TheBalancedValueImpactModel.KingsCollegeLondon,2012athttps://kclpure.kcl.ac.uk/portal/files/5675881/BalancedValueImpactModel_SimonTanner_October2012.pdf

VerhoevenD.2014.DoingtheSheepGood.InKBodeandAPLongley,AdvancingDigitalHumanities:Research,Methods,Theories.Palgrave,NewYork.

VerhoevenD,BurrowsT.2015.AggregatingCulturalHeritageDataforResearchUse:TheHumanitiesNetworkedInfrastructure(HuNI).InEGaroufallouetal.(eds),MetadataandSemanticsResearch:9thResearchConference,MTSR2015.BruceManchester,UK,September9-11,Proceedings,Springer,pp.417–423,2015.DOI:10.1007/978-3-319-24129-6_36.

VerhoevenD.2016.Asluckwouldhaveit.FeministMediaHistories,2:7-28.\

53

Appendix1VirtualLaboratoriesfundedbyNectar

Name

All-SkyVirtualObservatory

Alveo

BiodiversityandClimateChangeVirtualLaboratory

CharacterisationVirtualLaboratory

ClimateandWeatherScienceLaboratory

EndocrineGenomicsVirtualLaboratory

MicrobialGenomicsVirtualLaboratory

GenomicsVirtualLaboratory

HumanitiesNetworkedInfrastructure

IndustrialEcologyVirtualLaboratory

MarineVirtualLaboratory

VirtualGeoChemistryLaboratory

VirtualGeophysicsLaboratory

VirtualHazardImpactandRiskLaboratory

54

Appendix2CVLimagingfacilitypartners

Imagingfacility

AnimalMRIFacility,FloreyNeuroscienceInstitutes

AustralianCentreforMicroscopy&Microanalysis,USydney

AustralianSynchrotron

BiologicalOpticalMicroscopePlatform(MDHS),UoM

BiologicalResourcesImagingLaboratory,UNSW

AustralianCentreforNeutronScattering,ANSTO

CentreforAdvancedImaging,UQ

CentreforMicroscopyandMicroanalysis,UQ

CentreforMicroscopy,CharacterisationandAnalysis,UWA

Florey,MelbourneBrainCentre

FlowCore,MonashUniversity

MelbourneBrainCentreImagingUnit,UoM

MicroNanoResearchFacility,RMIT

MonashBiomedicalImaging

MonashBiomedicalProteomicsFacility

MonashInjuryResearchInstitute

MonashMicroImaging

MonashMicroImaging(AMREP)

UniversityofNewcastle,LightSheetMicroscopy

QueenslandBrainInstitute

SingleMoleculeScience,UNSW

RoyalChildren'sHospital

RoyalMelbourneHospital

55

StVincentsHospital

TheCliveandVeraRamaciottiCentreforStructuralCryo-ElectronMicroscopy

X-rayMicroscopyFacilityforImagingGeo-materials(XMFIG),Monash