ARIADNE: Final testing report

D13.5: Final testing report

Authors: Federico Nurra, InrapBernard Pinglier, Inrap

Ariadne is funded by the European Commission’s 7th Framework Programme.

Theviewsandopinionsexpressedinthisreportarethesoleresponsibilityoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheEuropeanCommission.

ARIADNED13.5:Finaltestingreport(Public)

Version:1.0(final) 24thJanuary2017

Authors: FedericoNurra,Inrap

BernardPinglier,InrapContributingPartners

RobertoScopigno,CNR-ISTI

BrunoFanini,CNR-ITABC

HollyWright,ADS

ARIADNED13.5:Finaltestingreport

3

TableofContents

DocumentHistory.......................................................................................................5

ExecutiveSummary....................................................................................................6

1 Introduction........................................................................................................8

2 RelatedWork....................................................................................................102.1 Firstreportonusers'needs(D2.1).............................................................................102.2 Secondreportonusers'needs(D2.2)........................................................................102.3 UserRequirements(D12.1)........................................................................................102.4 InfrastructureDesign(D12.2).....................................................................................102.5 InitialInfrastructureImplementationReport(D12.3)................................................112.6 InitialInfrastructureTestingReport(D12.4)..............................................................112.7 ServiceDesign(D13.1)................................................................................................122.8 InitialServicesImplementationReport(D13.2).........................................................122.9 InitialServicesTestingReport(D13.3).......................................................................122.10 Finalservicesimplementationreport(D13.4)............................................................13

3 Overviewofdesign,userequirementsandusers'specifications........................14

4 Servicesevaluated.............................................................................................184.1 AriadneCatalogue/Portal.........................................................................................184.2 ServicesprovidedontheAriadneCatalogue.............................................................18

4.2.1 Cataloguetextualsearch.....................................................................................................194.2.2 Spatialsearchanddisplay....................................................................................................194.2.3 Timelinesearchanddisplay.................................................................................................19

4.3 OtherServicescreatedwithintheARIADNEProject..................................................204.3.1 VisualMediaServices..........................................................................................................204.3.2 LandscapeServices..............................................................................................................21

5 Methodologyusedforevaluation......................................................................23

5.1 Scopeofthetesting....................................................................................................235.2 Generalaspectsonmethodology...............................................................................235.3 Testingscenarios........................................................................................................23

5.3.1 Evaluationquestionnaires...................................................................................................245.3.2 Groupsessions[workshop].................................................................................................26

5.4 Feedbacktotheimplementationteams....................................................................27

6 Evaluationresults..............................................................................................28

6.1 Generalfeedback.......................................................................................................28


4

6.2 CatalogueServices(46answers)................................................................................306.3 VisualMediaServices(56answers)...........................................................................366.4 LandscapeServices(21answers)...............................................................................396.5 HelpSection...............................................................................................................416.6 FinalEvaluation..........................................................................................................42

7 Conclusions.......................................................................................................47

8 References.........................................................................................................49

9 Tableoffigures..................................................................................................50

10 AnnexI–QuestionnairefortheevaluationoftheARIADNEServices.............51

11 AnnexII–TestingscenariosResults...............................................................55

11.1 BasicCatalogueServices............................................................................................5511.2 AdvancedCatalogueServices.....................................................................................5511.3 VisualMediaServices.................................................................................................5611.4 OwnCloudServices....................................................................................................5811.5 LandscapeServices.....................................................................................................58


5

DocumentHistory

Version Date Contributors Activity

0.1 November15th2016 FedericoNurra

BernardPinglier

Draftstructureofdocument

0.2 December6th2016 FedericoNurra Addedtestresults

Addedimages

0.3 December13th2016 BernardPinglier Drafttextreviewed

Modifiedlayout

0.4 December16h2016 RobertoScopigno Removedparagraph«Other

servicesprovided“asis”by

partners(notevaluated)»

0.5 January10th2017 FedericoNurra Added«Conclusions»

Addedtexton«AnnexII»

0.6 January12th2017 BrunoFanini Revised«LandscapeServices»

0.7 January20th2017 HollyWright QCReview

1.0 January24th2017 FedericoNurra Added«ExecutiveSummary»

Finaledit


6

ExecutiveSummary

This report is a deliverable (D13.5 Final testing report) of the ARIADNE project (“AdvancedResearch Infrastructure forArchaeologicalDatasetNetworking in Europe”) fundedunder theEuropeanCommunity'sSeventhFrameworkProgramme.ItpresentstheresultsofworkcarriedoutinTask13.4Acceptancetestingwithinworkpackage13DevelopingIntegratedServices.

ThisdeliverablereportsTask13.4,whichisiscentredontestingtheservicesproducedinTask13.2andverifyingtheircorrespondencetodesign(Task13.1),userequirements(Task12.1)anduser specifications (Task 2.1 and Task 2.2). New or improved additions were approved andpassedtothenextTaskorreturnedtoT13.2foramendments.Testingwascarriedoutwithinaspecific testing environment prepared by the partners in charge of the task. A number ofsimulatedcontextsweresetup,relevanttoarchaeologicalresearch,andthebehaviourofthesystem compared with user expectations. Overall compliance was also verified withrequirements, and the performance of the system in real test conditions. These tests wereperformed regularly and theiroutcomes reported to the leadpartnerofTask13.2,providingnecessary feedback. The testing specifications and the routine testing mechanism will bedetailedinTask13.1.

Methodology

Theevaluationhasbeenimplementedintwocomplementarydirections:

• Usingpredefinedtestingscenariosduringspecifictrainingworkshopwithselectedtesters

• Usingopenevaluationquestionnaireswithpowertesters

Theevaluationcyclefollowedseveralsteps.

Preliminarytasks:

• Creationofthegroupofevaluators

• Preparationoftheevaluationschemeandreports

• Evaluation

• Analysisofresults

• Productionofastructuredandcommentedfeedbacktotheimplementationteamand(ifnecessary), implementationoftherequestedchangesandproductionofarevisedversionoftheservice/tool

Thetaskleaderhadperiodiccontactswithservicesdevelopers,concerningthebugsandissuespointedoutbythetesters.

The aim of the questionnaire was related to a specific service, to determine whether theservicemeetstheexpectationsoftheusers.

Thisincluded:


7

• Theanswerstoprecisequestionsaboutusabilityoftheservice

• Opencommentsabouttheservice(usability,requestforimprovements)

• Areviewabouttheservice(similartowhatcanbefoundintheAppStore)

• Quantitative data about usage (e.g. number of downloads of an application,numberofclicks,numberoffilesuploaded,etc.)

Thepaneloftesters

AtINRAPapowertesterpanelwasformed,consistingof26people.Thepowertestersusedallthe services, answering the online questionnaire several times, following bug fixes and thedevelopmentofnewversions.TheonlyservicethatreceivedlessfeedbackthanexpectedwastheLandscapeService.Thisisduetohighertechnicalrequirementsneededbythetesters.

Eightexternaltesters,whoworkinthefieldof3Dmodelling,werecontactedtoevaluateVisualMediaServices.

The portal was tested by several partners. Some responded using the online questionnaire,others sent their informal feedback via email, or raised the critical issues during the«PortalUserGroup»Skypemeetings.

Results

Thequestionnaireanswersclearlyshowthattheserviceratingisveryhighfromtheevaluatorsand,overall,thattheperformanceshavemettheneedsofusers.Itshowssomedeficienciesinservices (such as the lackofmeasuringoptimized instruments, the lackof links between thedifferent services and the impossibility to download themajority of the elements present inservices).

Feedbackisextremelypositiveonthegeneralservices(average4.01/5.00)andthedevelopers’punctual interventions to fix thebugs foundby testershavecertainlyhelped toevaluate theservices as best as possible. Several testers have used the optional answers to suggestimprovementstothefacilities.


8

1 Introduction

This report is a deliverable (D13.5 Final testing report) of the ARIADNE project (“AdvancedResearch Infrastructure forArchaeologicalDatasetNetworking in Europe”) fundedunder theEuropeanCommunity'sSeventhFrameworkProgramme.ItpresentstheresultsofworkcarriedoutinTask13.4Acceptancetestingwithinworkpackage13DevelopingIntegratedServices.

As stated in the Description ofWork (DoW) of the ARIADNE Project, themain objectives ofWP13are:• To analyse, select, design and deploy the service components of the integrated

infrastructureinordertoimprovetheprovisionoftheonlineservicestotheresearchers• Totakeintoaccountinthedesignandthedeploymentfurtherenhancementsoftheservice

architectureasrequiredbytheadditionalimplementationofresultsfromWP14-WP17

ThetasksoftheWP13are:

Task13.1–ServiceDesignandSpecifications

Task13.2–ServiceImplementation

Task13.3–Long-termPreservationServices

Task13.4–AcceptanceTesting

Thisdeliverable reports Task13.4,which is centredon: testing the servicesproduced in Task13.2andverifiestheircorrespondencetodesign(Task13.1),userequirements(Task12.1)andusers’ specifications (Task2.1andTask2.2).Newor improved servicesmaybeapprovedandpassed to thenextTaskor returned to13.2 foramendments.Testingwillbecarriedoutonaspecifictestingenvironmentpreparedbythepartners inchargeofthetask.Theywillsetupanumberofsimulatedresearchcontexts,relevantforarchaeologicalresearch,andwillcomparethe behaviour of the system with the expected one. They will furthermore verify overallcompliancewithrequirementsandtheperformanceofthesysteminsimulatedrealconditions.Such testswillbeperformedregularly.Theiroutcomeswillbe reported to the leadpartnerofTask 13.2 for the necessary feedback. The testing specifications and the routine testingmechanismwillbedetailedinTask13.1.


9

Asshowninthefollowingfigure,takenfromD13.1,Task13.4isrelatedtothemajorityofworkpackagesandtaskswithintheARIADNEproject:

Figure1:ARIADNETasksandServiceDesign.


10

2 RelatedWork

2.1 Firstreportonusers'needs(D2.1)

Themainobjectivesofthisreportweretoprovideevidenceonuserrequirementsofkeytargetgroups (“users”) of theARIADNEprojectwith regard to archaeological research data, and tocontributetobuildingaprospectiveusercommunity.ThisinformationsupportedtheARIADNEproject in taking informed decisions regarding the specification of the e-infrastructure andservices so that the integrated infrastructure is developed in a way that corresponds toperceivedandactual researchneeds. Themandatewas toprovideevidenceon these issues,notablythroughcollectingfeed-backfromtheusercommunitiesbywayofusersurveys.

2.2 Secondreportonusers'needs(D2.2)

Themainobjectiveofthisreportwastoprovideadditional,moredetailedevidenceaboutuserrequirements of key target groups (“users”) of the projectwith regard to the ARIADNE dataportal.Apanelofabout25researcherswereaskedtodescribeindetailtheiruseofdigitaldataarchives, to evaluate existing archives and other portals and to highlight useful features oftheseportalswhichcouldserveas“goodpractice”whencreatinganewresearchdataportal.This information supports the ARIADNE project in taking informed decisions regarding thespecification of the e-infrastructure and services so they are developed in a way thatcorrespondstoperceivedandactualresearchneeds.Themandatewastoprovideevidenceonthese issues,notably throughcollecting feed-back fromtheARIADNEcommunitybywayofausersurvey.

2.3 UserRequirements(D12.1)

Themain objective of this report was to understand the current landscape from which theARIADNEinfrastructureshouldbecreated,inordertoinformthedevelopmentoftheARIADNEportal and services. This landscape included thedata,metadata, ontologies and vocabulariesavailable foruse,alongwithanyassociated issuesof licensingandaccess, informedbyusers’needs.

2.4 InfrastructureDesign(D12.2)

Themaingoalofthisreportwastospecifyaresourceintegrationanddiscoverymechanismforuse in ARIADNE. The resources to be integrated were datasets and collections, GIS data,metadataschemas,ontologiesandvocabulariesavailablefromtheprojectpartners,aswellasinstitutions outside the ARIADNE consortium. The deliverable provided an overview of theARIADNEarchitecture,includingasummaryoftheconformanceofthearchitecturetothedataandstandardsrequirementssetoutinD12.1,aswellasthespecificationsoftheServicesofthe


11

ARIADNE Infrastructure, presented in D13.1. Thiswas followed by a content analysis of themaincontenttypesdefinedinD12.1,andtheintegrationstrategyforthetwolevelsofcontent:the metadata integration, and the data integration. This will attempt to integrate selectedresources (datasets and/or metadata) from particular partners/data providers and providecross-search and access mechanisms to integrated resources, using a facetted search on“what”,“where”,“when”and“resourcetype”.

2.5 InitialInfrastructureImplementationReport(D12.3)

The main goal of this report was to present the initial infrastructure implementation ofARIADNE. The infrastructure is specifiedonD12.2 and included services such as the registry,vocabulary services, metadata enrichment services, preservation services, etc. with thefollowing components: a) an RDF store b) the first version of the portal c) the MOReaggregationinfrastructured)anElasticsearchcomponente)ametadataqualitymeasurementservicef)asetofenrichmentservicesTask12.3andthisreportfocusedonthedescriptionofcore services. The core services were described and implemented as individual servicecomponents in the overall architecture, and their complementary role was presented. Task12.3wasfollowedbyTask12.4Testing(whichisreportedinD12.4Initialinfrastructuretestingreport).Thiswas followedbyanotherroundof implementationandtesting inTasks12.5and12.6.TheAPIspecificationspresentedintheAnnexeswereespeciallyimportant,astheyallowdevelopers to build on the infrastructure and deploy services. The APIs presented are REST-basedandrequiredataformatssuchasJSON,XMLandRDF.

2.6 InitialInfrastructureTestingReport(D12.4)

ThetasksofWP12ImplementingInteroperabilityaimedtospecifyuserequirementsanddesignfortheARIADNEinfrastructureandtoimplementandtestthatdesign.Task12.4wascomprisedof testing the integrated infrastructurebuilt inTask12.3 Implementing Integration, basedonthe requirementsanddesignspecified inTask12.1UseRequirements and12.2 InfrastructureDesign. This document augmented the synthesis of requirements and design given in D12.2with the infrastructure implemented in D12.3, and tested the implementation against therequirements specified. Each requirementwas outlinedwith the infrastructural tools and/orservicesdesignedtomeetitandabriefoverviewoftheimplementationdocumentedin12.3.Recommendations were made based on this analysis. The general use requirements wereaddressedinthefirstinstance,followedbymorespecificrequirementsaddressingintegrationactivities.

The initial infrastructure implementation met many of the requirements specified. Someaspects not yet visible required further testing in Task 12.6 Final Testing Report. Someclarification of use requirements for the Preview Servicemay be necessary: D12.2 designs aservicethatpreviewssearchresults,asdescribedaboveandillustratedintheprototypeportal,asalist,onamapandtimeline(D12.2,24)butwhichcanalsopreviewindividualrecords.D12.3discussed the implementation of a Preview Service through which researchers can previewtheirowndatasets inanumberof formats,whichseemstobemorecloselyalignedwith the


12

Registry (17). The Support Portal, which includes documentation and user guidance, wasapproachedholisticallyinordertoprovideclearandcoherentguidancetoavarietyofusersatdifferent points of interactionwith the infrastructure. Documentationwas required for eachmajor component of the infrastructure. The Registry input tool provides the necessaryfunctionality to enabledepositingofmetadata. Theuser experienceof theRegistry couldbeimprovedandmorecloselyguidedwithchangesindesign.Atableofrecommendationsfortheinfrastructureisincludedattheendofthisdocument.

2.7 ServiceDesign(D13.1)

This document specified the service design of the ARIADNE Portal, and provided a commonvision,auserperspectiveonthefunctionality,andaframeworktoidentify,discussandvalidatetherequirementsfortheunderlyingtechnicalservices.Assuch,theaudienceofthisdocumentwasbothtechnicalandnon-technical.

Thisservicedesignwascreatedusingthefollowing input:Task2.1Surveyofusers’needsandcommunity building, Task 12.1 Assessment of use requirements, an examination of existingportalservicesandTask13.4Acceptancetesting.Theservicedesignprovidedacommonvisionformulated in terms of principles, derived from themission statement in the Description ofWork.

2.8 InitialServicesImplementationReport(D13.2)

ThisdocumentgivesanoverviewoftheinitialimplementationoftheservicesoftheARIADNEinfrastructure,whicharetheobjectiveofWorkPackage13(WP13).WP13wasinformedbytheoutput of WP2 (mainly Task 2.1 User needs and community building and Task 2.2 SpecialInterestGroups,andparallelsthedataintegrationeffortinWP12.Moreover,theservicesmadeavailablewithinWP13 also incorporate those developedwithinWP14 throughWP16. Theseserviceswill includefunctionalitythat isalreadyavailable,andwillbeofferedtocommunitieswhomaynotcurrentlyhaveaccesstoit,butwillalsoincludenewfunctionalitycreatedadhocby thepartnerson thebasisof the requirements thatwerecollectedat thebeginningof theproject. Themainmodality of the implementation of this new functionality as a web-basedservice,althoughothertypesofservices(localtools,guidelines)wereintegratedaswell.

2.9 InitialServicesTestingReport(D13.3)

D13.3presentedtheinitialresultsofTask13.4(T13.4),withinWorkPackage13(WP13)

Thereportfocusedonthefollowingaspects:

• Overview of design, use requirements and user's specifications, the compliance withwhich the services have to be tested (see D13.1, D12.1 and D2.2 for amore detailedpresentation).


13

• Servicestobeevaluated:functionaldescriptionandprogressreportforimplementationofthese(seeD13.2foramoredetailedpresentation).

• Methodologyusedforthetests.

• First testing results. Informal tests were conducted by Inrap among a community ofinternalpowerusers,showingagreatinterestintheserviceprovidedforvisualisationof3D images, but also the need for enhanced visualisation and analysis tools, especiallymeasurementtools.

2.10 Finalservicesimplementationreport(D13.4)

This Deliverable presented the final state of integration and implementation of servicesdevelopedinthecontextoftheARIADNEinfrastructure.Itpresentedindetailtheorganisation,thefeaturesandthedesignofalltheserviceswhichformtheARIADNEinfrastructure

ThisdocumentpresentedthefinallistofservicesdesignedandimplementedfortheARIADNEinfrastructure,whichwastheobjectiveofWorkPackage13(WP13).

WP13wasinformedbytheoutputofWP2(mainlyTask2.1Userneedsandcommunitybuildingand Task 2.2 Special Interest Groups), and parallels the data integration effort in WP12.Moreover,theservicesmadeavailablewithinWP13alsoincorporatedthosedevelopedwithinWP14throughWP16.

These services include functionalities that were already available, and are offered tocommunitieswhomaynothadaccesstothem,butdoalsoincludenewfunctionalitiescreatedadhocbythepartnersonthebasisoftherequirementsthatwerecollectedatthebeginningoftheproject.Themainmodalityoftheimplementationofthesenewfunctionalitiesistoprovideweb-basedservices,althoughothertypesofservices(localtools,guidelines)wereintegratedaswell.


14

3 Overview of design, use requirements and users'specifications

Theusers'needsaresummarisedasfollows,derivedfromD2.2:

1.OverallfocusoftheARIADNEprojectisondatadiscoveryandaccessservices.

2. The primary focus of the design and interfaces of the data portal should be anoverview ofwhat data is accessible, including statistical information on quantity,types,anddistribution(e.g.country/area,period).

3. The portal should focus on the European/international dimension. Lack ofunderlyingresources(percountry,typeofdata,etc.)shouldnotbeseenasadeficit,but used to promote data mobilisation (e.g. implementation of national dataarchives).

4. Added value should also be created through linking data and publicationresources not held within the ARIADNE Registry (e.g. metadata of documentarchivesandopenaccesspublishers).

5. Linked Open Data (LOD) can play a core role for value generation, but furtheruptake of LOD principles by archaeological institutions and projects must beencouraged.

6.Inthedevelopmentofthedatasearch,accessandotherservices,membersoftheuser community must be thoroughly involved and regular feedback onimplementedsolutionssoughtbythewidercommunity.

7. User-focused development of the portal services and applications (relevance,usability,user-friendliness)shouldbeatthetopoftheproject’spriorities.

8. Services for websites for research communities in particular subjects orgeographicregions(e.g.alertsonrelevantdatasets)couldgreatlyexpandthereachofthedataportaland,inturn,promotefurtherdatamobilisation.

9. Full exploitation of the data resources (incl. metadata, conceptual knowledge)shouldbeenabledby interfaces forexternalapplications (e.g.awell-documentedAPI,OAI-PMHtarget,SPARQLendpoint).

10.Supportofe-research/scienceshould,inthefirstinstance,beprovidedthroughintegratingaccesstodataresourcesandbypointinguserstoexistingtoolsfordataextraction,processingandanalysis.


15

Theuserequirements(D12.1)refertotherequirementsforthedesignandspecificationofthesubsequenttoolsandservicesnecessaryforintegration.D12.1producedrecommendationsforDatasets;MetadataStandards,SchemasandVocabularies;andAccessandSharingPolicies.

Datasets• Site andmonumentdatabases:Most European countries and/or regionshave them,

andcombiningthemmaybeusefulforcross-bordersearchingandgeo-location• Interventionactivity:Mayhavemultipleactivitiesassociatedwithageo-locatablesite,

whichmayallowlinkingofvariousactivitiestoasinglesiteormonument• Fieldworkdatabases:Usuallytoodiverse,soindividualdatabasesmaynotbeusefulfor

integration, but may be worth linking to intervention activities for bibliographicdiscovery

• Othercategoriesarequitespecific,butmaybeusefulforintegration:o ScientificDatabaseso ArtefactDatabaseso BurialDatabases

Balancedataqualityandquantity:specifyrequirementsthatdatasetshavetomeetinordertobeintegrated,preferablyusingformalcriteria.Therelationshipsbetweenthetypesofdataavailablefromthecontentprovidingpartnersandthe recommended integration activity to be designed within D12.2 are set out in the tablebelow.

MetadataStandards,SchemasandVocabularies• Theuseofinternationalstandardsforthedocumentationofexcavationsand

monumentssoastorenderittransparentandcomparableand,aboveall,makethemmoreinteroperable.

• Freeaccesstotools,particularlyfordatamapping,tomakeiteasiertoincorporatethesestandards,andofferingthemeansandguidanceforarchaeologiststodeposittheirdigitalrecordsinanappropriatearchive.

• Thesustainabilityofdigitaldatasetsmustalsobehighontheagenda


16

TherelationshipsbetweenthewishesandconcernswithregardtometadataandtherecommendedtoolstobedesignatedordesignedwithinD12.2aresetoutinthetablebelow:

Tableshowingthewishesandconcernswithregardtodatastandards,categorisedbythetypeofschemaorvocabularywhichmayaddressthewishes,andthetoolswhichmayaddresstheconcerns.The+signifiesthelevelofimportance.AccessandSharingPolicies• Acommonmethodofdatacitationshouldbeestablishedforadoptionbypartners,and

promotedbyARIADNEtothearchaeologicalresearchcommunity.Academicrecognitionisanimportantmotivationforencouragingresearcherstoshareaccesstotheirdatasets

• AllocationofDOIsortheequivalenttodatasetsingestedtotheARIADNEinfrastructureshouldbeinvestigated.Thesystemusedshouldbecapableofidentifyingsub-setswithincollections.Persistentidentificationofdatasetsisimportantinunderpinningdatasharinganddatacitation

• Content itself (databases, document archives, images, 3D models, etc.) should beprovided to ARIADNE by content partners using the Creative Commons license suite(version4.0 ispreferred)under licensepermissionsagreedwith thecontentowner.CCBY is recommended for open access. CC BY SA or CC BY SA NC licenses may also beapplicable

• A Collection description (of the whole collection and subsets within the collection)should be published under a CC BY license for each dataset ingested to the ARIADNEinfrastructure

• Metadata records shouldbepublishedunderaCC0 license – toenable integrationofmultipledatasetswithinthemetadatarepository,supportresourcediscoveryandenablelinkedopendata

The ServicesDesign (D13.1) introduces the functionality of the services, derived from a userperspective,andthusgivesanimportantinputforacceptancetesting.Itpresentsasetofusecasesrepresentingvariousrealsituationsencounteredbymembersofthearchaeologicalcommunity:


17

• Search andexplore theRegistry to look for archaeological datasets to bepreviewedand/ordownloaded

Theuse case is divided in several steps (navigation, entering searchparameters,displaying the results) and relates to several services e.g. portal access, timelinesearch,geographicalsearch,displayoftheresults,etc.

• PreviewdatatodeterminetherelevanceofthisdataforaspecificresearchworkThepreviewserviceandfunctionalitydependsonthenatureofthedata.

• Accessdatabydownloadingitforfurtherprocessing• Deposit data in the Ariadne infrastructure to allow it to be browsed and re-used by

fellowresearchersThe service requires a set of guidelines for depositing data and an archivecompatiblewiththedatatobedeposited.

• Search and access the service registry to discover tools and knowledge to supportresearch/datamanagementactivities

• PrepareandregisteranewcollectiontotheRegistry• Enrich VisualMedia Documents from one of the visual documents (3Dmodels, RTI

images, high resolution images, terrain models) stored in one of the cataloguesassociatedtoAriadnePortal

Provideavisualizationwindowshowingthevisualservicesthatcouldbeappliedtoavisualmedia

• Manage accounts to allow a securitymanager to handle and control authenticationandaccess

The security manager may have an overview of the users and eventually change theirattributes.


18

4 Servicesevaluated

Thissectiongivesanoverviewoftheservicesevaluated.

4.1 AriadneCatalogue/Portal

ARIADNEprovides services forarchaeologists toenableaccess to the research infrastructure.TheARIADNEPortalprovidesthemainpointofaccessforsearchingandbrowsingdatasetsandnewservicesforprocessingandpublishingarchaeologicaldatasetsonline.

A beta version of the ARIADNE portal is available at http://portal.ariadne-infrastructure.eu/.TheportalbringstogetherexistingarchaeologicalresearchdatasetsfromARIADNEpartnerssothat researchers can browse and access the various distributed datasets for use in theirprojects.

Theportal also provides a point of access for the new services developedby the project forvisualmediaandlandscapedatasets.

Figure2:ARIADNEPortal-HomePage

4.2 ServicesprovidedontheAriadneCatalogue

The ARIADNE Catalogue is based on the ACDM (Ariadne Catalogue Data Model), whichdescribes the archaeological resourcesmade available by the ARIADNE infrastructure to theresearchers wishing to access and use them. The current version is ACDM 2.6, delivered onSept.26th,2015.


19

4.2.1 Cataloguetextualsearch

TheUsercanbeginanewsearchbyusingthesearchbar.Alternatively,userscanbrowsetheCatalogueusingtheWhere/When/Whatfacets.

Figure3:ARIADNEPortal–Searchwindow

4.2.2 Spatialsearchanddisplay

The firstbrowse facet isadirect link to theARIADNEGeographicSearchMap.Userscan findresourcesbasedongeographiccriteriabyclickingonthespecificareaofinterest.

Figure4:ARIADNEPortal–Geosearchwindow

4.2.3 Timelinesearchanddisplay

Thesecondbrowsefacet istheARIADNETimeline.Userscanusetemporalcriteriatoretrieveresourcesconcerningaspecificperiod.


20

Figure5:ARIADNEPortal–Timelinesearchwindow

4.3 OtherServicescreatedwithintheARIADNEProject

4.3.1 VisualMediaServices

TheARIADNEVisualMediaServiceprovideseasypublicationandpresentationon thewebofcomplexvisualmediaassets. It isanautomaticservicethatallowstheuploadofvisualmediafilestoanARIADNEserverandtotransformthemintoanefficientwebformat,makingthemreadyforweb-basedvisualisation.

http://visual.ariadne-infrastructure.eu


21

Figure6:VisualMediaServices–HomePage

4.3.2 LandscapeServices

LandscapeServicesforARIADNEareasetofresponsivewebservicesthatincludelargeterraindataset generation, 3D landscape composing and 3Dmodel processing, leveraging powerfulopen-source frameworks and toolkits such asGDAL,OSGjs,OpenSceneGraph and ownCloud.Themain components include: the cloud service, the terrain generation service, the terraingalleryandthefront-endwebcomponentforinteractivevisualisation.

http://landscape.ariadne-infrastructure.eu


22

Figure7:LandscapeServices–HomePage


23

5 Methodologyusedforevaluation

5.1 Scopeofthetesting

Testingwascarriedoutontheservicesdescribedintheprevioussection.

Noevaluationwasconducted forpreviouslyexistingservicesoperatedbypartnersandmadeavailable through ARIADNE, only the newly provided access to these services through theARIADNEPortalhavebeenevaluatedasapartoftheARIADNEportalservice.

5.2 Generalaspectsonmethodology

Theevaluationhasbeenimplementedintwocomplementarydirections:

• Using predefined testing scenarios according to the following matrix, during specifictrainingworkshopswithselectedtesters

• Usingopenevaluationquestionnaires,bothwithpowertesters(selectedbyInrap,leaderofTask13.4andbyCNR,leaderofWP13)

Theevaluationcyclefollowedseveralsteps:

• Preliminarytasks:

o Creation of the group of evaluators, ~30 power testerswithin Inrap and ~10power testers outside (e.g. University of Cambridge, University of Lund,UniversityofTurin,etc.)

o Preparationoftheevaluationschemeandreports

• Conductingtheevaluation

• Analysingtheresults

• Producing a structured and commented feedback to the implementation team and (ifnecessary), and implementationof the requested changes andproductionof a revisedversion of the service/tool. The task leader has had periodic contacts with servicesdevelopers,concerningthebugsandissuespointedoutbythetesters.

5.3 Testingscenarios


24

Eachtestingscenariowasappliedtooneorseveralservicesforwhichitwasrelevant,accordingtothefollowingmatrix(seeAnnexIIfortheresults):

SERVICES

CATALOGUE VISUALMEDIASERVICES LANDSCAPESERVICES

1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 3.3

CatalogueSearch

SpatialSearch

TimelineSearch

3DModels RTI HighRes

ImagesOwnCloud 3DTerrain Terrain

Gallery

ACTIONS

Navigate

Search

View

Edit

GeoSearch

TimelineSearch

Upload

Conversion

Manage

Download

Embed

5.3.1 Evaluationquestionnaires

Contentofthequestionnaires

Theaimofthequestionnairerelatedtoaspecificservicewastodeterminewhethertheservicemettheexpectationsoftheusers.

Thisincluded:− Theanswerstoprecisequestionsaboutusabilityoftheservice


25

− Opencommentsabouttheservice(usability,requestforimprovements)− Areviewabouttheservice(similartowhatcanbefoundintheAppStore)− Quantitativedataaboutusage(e.g.numberofdownloadsofanapplication,numberof

clicks,numberoffilesuploaded,etc.)

Themultilingual test formswerebeen createdusing the “Google Forms”platform, followingtheschemaattachedinAnnexII.

Figure8:Testingquestionnaire(GoogleForm)

Translation

The original questionnaireswere created in English and translated into French and Italian toprovide them to wider communities. The feedback received has been fully translated intoEnglish, including the comments that were useful for evaluation purposes, which andperiodicallytransferredtoservicedevelopers.

Operation

Thequestionnairewassenttoaselectedsetofusers,previouslyidentifiedas:


26

− PersonswhoshowedaninterestforARIADNE(respondentstothesurveyconductedinWP2)

− Users of the specific service willing to answer questions online (tester panel withinInrap)

Thequestionnaire,accordingtotheschemepresentedinAnnexII,wasproducedonaGoogleFormandwillbesenttothelistoftesters.

Analysisofthedata

Theanalysiswasmadebythetaskleader.

TheresultsoftheperiodicanalysiswerecommunicatedbymailtotheWPleader,whohelpedtofixvariousbugsonthesystem(e.g.October2016:tiffbugfixonVisualMediaServices.«Tiff-especiallyLidar-imageprocessingwasplaguedbytifflibsupport,andhasbeenfixed»)

Particular attentionwasgiven to theglobal acceptanceand implementationof theproposedservices.

Attention was also focused on the proper functioning of services and the speed of theoperationscarriedoutbytesters.

5.3.2 Groupsessions[workshop]

AtestingworkshopwasorganisedinParisonMay30th2016.Itwasagroupsessionwithalmost30usersusingthedifferentservices.Itwasusefultocollectinformationabouttheusageoftheservice.Thefirstphaseconcernedafreeusageoftheservicesandafirstsetofanswers.Thesecond phase followed a web tutorial, which explained the detailed service functions.Afterward,a secondsetofanswerswascollected.Theworkshopparticipantshavealsobeenperiodicallyinvolvedinremotetestingsessions.


27

Figure9:TestingWorkshop–Paris,May30th2016

5.4 Feedbacktotheimplementationteams

During the final year of project (after the services beta version release), there have beenperiodiccontactsbetweenthetask leaderandservicedevelopers.Thetask leader (Inrap)didnotadheretothequarterlyfrequencyplannedonD13.3InitialServicesTestingReport,astheflowofanswersfromthetesterswasnotsteady.Nevertheless,thecontactsweremadewhenappropriaterelatingtotheissuesandproblemsencounteredbytesters.


28

6 Evaluationresults

Thischapterpresentstheresultsoftheonlinequestionnaires,withthesupportofexplanatorygraphicsbasedon123feedbackreportsfromabout50participants,structuredasfollows:

- AtINRAPapowertester’spanelwasformed,consistingof26people.Thepowertestersused all the services, answering the online questionnaire several times, following bugfixes and the development of new versions. The only service that has received lessfeedback than expected was the Landscape Service. This is due to higher technicalrequirementsneededbythetesters.

- Eightexternaltesters,whoworkinthefieldof3Dmodelling,werecontactedtoevaluateVisualMediaServices.

- The portal was tested by several partners. Some responded using the onlinequestionnaire,otherssenttheirinformalfeedbackviaemail,orraisedthecriticalissuesduringthe«PortalUserGroup»Skypemeetings.

6.1 Generalfeedback

Howwasthewebsitereached?


29

Wastheservicedescriptionprovidedonthewebclear?

Ifnot,whynot?(mostrelevantanswers)

• Notabletofindanaccuratedescriptionofthefeatures

• LackofMultilingualapproach

• Itwouldbehelpfultohavemoreinfo

Wastheprovidedservicespeedsatisfactory?


30

ServicesUsed:

6.2 CatalogueServices(46answers)

KindofSearch(multiplechoices):


31

Howmanysearcheswerecarriedout?

Wastheresultsvisualisationsatisfactory?

Ifnot,whynot?(mostrelevantanswers)

• Lackofdata

• Difficulttounderstandwhichtermtousefortextualsearch

• Searchby“Subject”isunclear

• Lackofthesauriexplanation


32

Werethepagenavigationcommandsclear?

IfNot,whynot?(mostrelevantanswers)

- Howtomoveonthemapwiththemouseisnotclearatall

- Difficulttounderstandthedifferencebetweenthetwoareasoftextualresearch,“startanewsearch”ontopand“newsearch”intheresultspage

- LackofMultilingualapproach

WereLinksusedtoreachtheoriginaldata?


33

IfYES,didtheyworkproperly?

Weretheresultssatisfactory?


- BrokenLinks

- MissingLinks


34

Werefiltersusedforthesearch?

Whichfiltershavebeenused?


35

Didthefilterworkproperly?

Wasiteasytofiltertheresults?


- Selectionnotsatisfactory

- Textualsearchdoesn’tworkproperly

- Timelinesearchisn’tintuitive


36

6.3 VisualMediaServices(56answers)

Whichdatawereuploaded?

Whatwasthetotalsize(inMb)oftheuploadeddata?

Resultsbetween1.6Mband122Mb(average:29.5Mb)

3DModels:

Wasthe3Dmodelvisualisationsatisfactory?



37

- Wrongvisualisationoflongtitleswithinthecatalogue

- Problemwithplyfilesover50Mb

- Colourprojectionneedssomeimprovement

Wasthenavigationspeedofthemodelsatisfactory?

Wasthemodelqualitysatisfactory?


- Nogeoreferencing

- Nounitmeasure

- Lackofmetricprecision


38

RTIImages

WastheRTIvisualisationsatisfactory?

100%“Yes”

Wastheresponsespeedofthelightsatisfactory?

100%“Yes”

Wasthereflectancemodelqualitysatisfactory?

100%“Yes”

HRImages

WastheHRIvisualisationsatisfactory?

100%“Yes”

Wasthezoomspeedsatisfactory?

100%“Yes”

Wastheimagedefinitionqualitysatisfactory?

100%“Yes”


39

6.4 LandscapeServices(21answers)

Landscape

Wasthegalleryseen?

100%“Yes”

Wasthemodelvisualisationsatisfactory?


- IssueswithSafariBrowseronMacOS

- Themovementsonthemodelaren’tintuitive

Haveanyprocessbeenrun?

100%“Yes”

Whatkindofoutputwaschosen?(Multiplechoice)


40

NB:theDAEColladaexportsupportisafuturedevelopment

Whatkindofresolutionwasused?

WasanareadefinedwithanESRIshapefiles?


41

IfYES,areyousatisfied?

100%“No”


- Shapefilecutdoesn’twork

6.5 HelpSection

Wasthehelpsectionused?

IfYES,wasithelpful/satisfactory?


42

IfYESbutitdidnotsolveissues,why?(mostrelevantanswers)

- Difficulttofindthe“helpsection”

- Notuseful

- Explanationsnotclear

Haveproblems/bugsbeenfoundwhenusingthesystem?

Ifproblems/bugswerefound,havetheybeenreportedtodevelopers?

100%“No”

NB:issueswerereportedtodevelopersthroughtheT13.4leaderandwerealwaysaddressed

6.6 FinalEvaluation


43

Wastheserviceperformancesatisfactory?


- Lackofmetrics

- Visualisationproblems

- NecessityofhavinganaccounttouploaddataonLandscapeServices

- Lackofconsistencybetweenservices

- Noteasilyusable

- Impossibletodownloadsearchresults

- LackofmetadatamanagementonVisualMediaService

- Lackofexporttools

- Lackofmeasurementtools

- Lackofanalysistools

Wasthesystemusedwhileconnectedto:


44

GlobalJudgmentontheexperience:

Globalaverage:4.01/5.00

Catalogueaverage:3.90/5.00

VisualMediaaverage:4.16/5.00

LandscapeServicesaverage:4.02/5.00

Arethereanyfurthersuggestionsforthedevelopers?


45

IfYES,what?(mostrelevantanswers)

- Optiontodownloadthesearchresultsonthecatalogue

- OptiontocreateaRSSfeedrelatedtoaspecificquery

- Optiontocreateemailalertrelatedtotheresourcesonthecatalogue

- Implementationofstatisticaltools

- Addnewkindof“sortby”inthecatalogue

- Improvetheergonomyofthesystem

- Accesstothemetadataforthe3Dmodels

- NewwidgetsandtoolsforVisualMedia(lights,sections,analysis…)

- Improvetheharmonisationoftheservices

- Addalegendonthemapsearch

- IncreasetheusableformatsonVisualMediaService

- NeedcomputationtoolsandspatialanalysistoolsfortheLandscapeServices

- AzoomfunctionforDEMdisplayafterbeingprocessedwouldbeuseful

- Improvethecolourprojection

- Possibilityofdefiningastartingview/coordinatesystem(VisualMedia)

Whichotherdatatypeshouldbemanagedbytheservices?

- SpatialdatafromtheCatalogue

- 3Dspatialdata

- Items’Metadata


46

- GeoDB

- LinkbetweenVisualMediaItems,LandscapeModelsandCatalogue

- RawdataofphotogrammetryTerrainmodelcreation

- Archaeologicalimagescollection

- 3Dshapefiles

- 360°panoramasin.movformat


47

7 Conclusions

The questionnaire’s answers clearly show that the service rating is quite high from theevaluators (§7.6 “Global Judgment on the experience”) and, overall, that the performanceshavemettheneedsofusers(§7.1“Wasthespeedoftheserviceprovidedsatisfactory?”),whileshowingsomedeficiencies inservices(suchasa lackofoptimisedmeasurement instruments,the lackof linksbetween thedifferent servicesand the inability todownload themajorityoftheelementspresentinservices).

It was taken into account in the results analysis, that most of the testers used broadbandconnections(>10Mbps).

Onlyasmallnumberoftestersusedthehelpsectionowingtothefactthat80%ofusersfoundnoproblemsintheuseofservices.

Ingeneral,theoriginsofthetestersweremostlyfromFrance,asINRAPwastheTaskLeaderofthe T13.4 and more than half of the responses came from testers who followed a specificworkshopinParis.

Accordingtotheusers,theservicesareingeneraleasytouse.

In detail, the largest number of answers related to the Visual Media Services; then to theCatalogue/PortalandfinallytotheLandscapeServices.

As for the Catalogue, it is evident that most testers have used the three different kinds ofsearch jointly (Text, Geo, Timeline - 64%) and only a small number used a single searchfunction.Theaveragenumberofsearcheswasnotveryhigh(~14peruser)butthiswasduetothe need to test the services in a reasonable amount of time in order to provide usefulfeedback.Thesearchservice in theCatalogue (Portal)was triedbymostof the testers (73%)whoevaluateditpositivelywithanaverageof3.90/5.00.Howtousetheservicewascleartomostofthetesters,andthenavigationandsearchtoolswereeffectiveandfunctional.

TheVisualmedia servicesweremainlyused foruploadingandviewing3Dmodels (56%)andHigh Resolution (HR) images (26%). This is probably due to a lack of knowledge about theprocesses used to create and use RTI images. The general acceptance of the Visual MediaServiceswas thehighest of the three services evaluated: 4.16/5.00. The satisfactionwith 3Dmodel users was quite high (71%) with peaks related to service speed (85%) and imagerenderingquality(81%).Theproblemsencounteredwererelatedtodifficultieswithuploadinglargesize3Dmodels(>50Mb)andthecolourrenderingofmodels.RegardingHRimages(13answers) and RTI images (10 answers) the global satisfaction was 100% approval, andemphasises,onceagain,theneedforarchaeologiststobeabletotakeadvantageoffreetoolsforimagemanagement.

TheLandscapeservices,despitehavingfewerevaluationsthantheothertwo,however,hadahighapproval rating (4.02/5.00). Thedisplayquality satisfied90%of the testers,withawideuse of terrain service for the construction ofmodels in JSONWebGL (navigable on common


48

browsers,bothdesktopandmobile).ThesecondchoicewastheOBJformat,afreeformatfor3D model interchange. Most of the testers used a maximum resolution (60%) or mediumresolution (27%). One of the service tools (cutting with ESRI shape files) was still indevelopment,soitwasnotpossibletoevaluatethisfeature.

Toconclude,feedbackwasextremelypositiveonthegeneralservices(average4.01/5.00)andthedeveloperspunctualinterventionstofixbugsfoundbytestershelpedtheevaluationoftheservices to be the best possible. Several testers used the optional answers to suggestimprovements to the facilities: themost requested feature was the ability to download theresults from the Portal and to download objects from the Visual Media public galleries,followedbyrequestsrelatedtoimprovedmetadatamanagement(forallservices).Specificallyfor manipulating 3D models in visual media, which often requires the ability to use otherformats than .ply. Within the Landscape Services the lack of computation tools and spatialanalysistoolswashighlighted.Finally,fromtheperspectiveofthetesters’proposalsformoreservicestobedevelopedinthefuture,theyrequestedtoolsforthetreatmentof3DGISvectordata,processingof imagecollectionsandthemanagementofphotogrammetryterrainmodelcreationrawdata.Althoughproposedonlybyonetester,thesuggestionforthedevelopmentofapplicationsforthemanagementof360°panoramasin.movformatwasalsointeresting.


49

8 References

ARIADNE,2012,Ariadne-Ariadne,http://www.ariadne-infrastructure.eu/

ARIADNE,2014,D2.1“Firstreportonusers’needs”,http://www.ariadne-infrastructure.eu/Resources/D2.1-First-report-on-users-needs

ARIADNE,2014,D12.1"UserRequirements",http://www.ariadne-infrastructure.eu/Resources/D12.1-Use-Requirements

ARIADNE,2014,D13.1"ServiceDesign",http://www.ariadne-infrastructure.eu/Resources/D13.1-Service-Design

ARIADNE,2015,D2.2"Secondreportonusers'needs",http://www.ariadne-infrastructure.eu/content/view/full/1188

ARIADNE,2015,D12.2"InfrastructureDesign",http://www.ariadne-infrastructure.eu/Resources/D12.2-Infrastructure-Design

ARIADNE,2015,D12.3"InitialInfrastructureImplementationReport",http://www.ariadne-infrastructure.eu/Resources/D12.3-Initial-Infrastructure-Implementation-Report

ARIADNE2015,D13.2"Initialservicesimplementationreport",http://www.ariadne-infrastructure.eu/Resources/D13.2-Initial-services-implementation-report

ARIADNE2016,D13.3"Initialservicestestingreport",http://www.ariadne-infrastructure.eu/Resources/D13.3-Infrastructure-services-testing-report

ARIADNE2016,D13.4"Finalservicesimplementationreport"[v.0.4-draftversion]


50

9 Tableoffigures

Figure1:ARIADNETasksandServiceDesign. 9

Figure2:ARIADNEPortal-HomePage 18

Figure3:ARIADNEPortal–Searchwindow 19

Figure4:ARIADNEPortal–Geosearchwindow 19

Figure5:ARIADNEPortal–Timelinesearchwindow 20

Figure6:VisualMediaServices–HomePage 21

Figure7:LandscapeServices–HomePage 22

Figure8:Testingquestionnaire(GoogleForm) 25

Figure9:TestingWorkshop–Paris,May30th2016 27


51

10 AnnexI–QuestionnairefortheevaluationoftheARIADNEServices

TESTERDATA

FirstName

[Text]

LastName

[Text]

Institution

[Text]

Country

[Text]

START

Howwasthewebsitereached?

[multiplechoice]

Google DirectLink AriadneSite Other

Wastheservicedescriptionprovidedonthewebclear?

YES NO

IfNO,why?

[Text]

Wastheprovidedservicespeedsatisfactory?

YES NO

IfNO,why?

[Text]

Usedservices

Catalogue VisualMedia Landscape

CATALOGUE VISUALMEDIA LANDSCAPE


52

Kindofsearch Whichdatahavebeenuploaded? Hasthegallerybeenseen?

[multiplechoice] [multiplechoice] YES NO

Text Geo Timeline 3DModels RTIImages HRImages Wasthemodelsvisualizationsatisfactory?

Howmanysearcheshavebeendone? Whatwasthetotalsize(inMb)oftheuploadeddata? YES NO

[multiplechoice] [Integer] IfNO,why?

1-10 10-20 20-50 >50 [Text]

Wastheresultsvisualizationsatisfactory? Doyouwanttoanswertospecific

questionsabout3Dmodels? Haveanyprocessbeenrun?

YES NO YES NO YES NO

IfNO,why? Whatkindofoutputhasbeenchosen?

[Text] 3DMODELS [multiplechoice]

Werethepagenavigationcommandsclear?

Wasthemodelvisualizationsatisfactory?

1 2 3 4 5 6

YES NO IfNO,why? Whatkindofresolutionhasbeenused?

IfNO,why?

Wasthenavigationspeedofthemodelsatisfactory?

Low Medium High

[Text] IfNO,why? Hasbeenanareadefinedwithashpfile?

Werelinksusedtoreachtheoriginaldata? Wasthemodel

qualitysatisfactory?

YES NO

YES NO IfNO,why? IfYES,areyousatisfied?

IfYES,didtheyworkproperly? YES NO

YES NO DoyouwanttoanswertospecificquestionsaboutRTIImages? IfNO,why?

Weretheresultssatisfactory? YES NO [Text]

YES NO HavefilesinOWNCloudbeen

uploaded?

IfNO,why? RTIIMAGES YES NO

[Text] WastheRTIvisualizationsatisfactory?

Werefiltersusedforthesearch? IfNO,why? OWNCLOUD

YES NO Wastheresponsespeedofthelightsatisfactory?

Whichdatatypes

havebeenuploaded?

IfNO,why? [multiplechoice]

ADVANCED

Wasthereflectance

modelqualitysatisfactory?

DEM Raster Shp

Whichfiltershavebeenused? IfNO,why?

Whatisthetotalsize(inMb)oftheuploadeddata?


53

[multiplechoice] [Integer]

Text Geo Timeline Doyouwanttoanswertospecific

questionsaboutHRImages? Wastheservice

globallysatisfactory?

Didthefilterworkproperly? YES NO YES NO

YES NO IfNO,why?

IfNO,why? HRIMAGES [Text]

[Text] WastheHRIvisualizationsatisfactory?

Hasitbeensimpletofiltertheresults? IfNO,why?

YES NO Wasthezoomresponsespeedsatisfactory?

IfNO,why? IfNO,why?

[Text] Wastheimagedefinitionqualitysatisfactory?

IfNO,why?

HELPSECTION

Hasthehelpsectionbeenused?Hasitbeenresolutive/helpful/satisfactory?

YES NO

IfYESbutitdidnotsolveissues,why?

[Text]

Haveproblems/bugsbeenexperimentedusingthesystem?

YES NO

Incaseproblems/bugshavebeenexperimented,havetheybeenreportedtothedevelopers?

YES NO

IfYES,hasananswerfromthedevelopersbeenreceived?

YES NO

FINALEVALUATION

Havebeentheserviceperformancessatisfactory?

YES NO

IfNO,why?


54

[Text]

Havebeenthesystemusedwhileconnectedto:

[multiplechoice]

Highspeednetwork(>10MB) Ordinarynetwork(3-10MB) Lowspeedconnection(<3MB)

Globaljudgementontheexperience:

(*****)

Experiencedescription(optional)

[Text]

Areanysuggestionsforthedeveloperspossible?(Possiblesuggestionsforthedevelopers)

YES NO

IfYES,what?

[Text]

WhichotherdatatypeshouldbemanagedbytheServices?(Specifydatatype)

[Text]

WhichvisualizationorinteractionfeaturesshouldbeconsideredwhileextendingtheVisualMediaService?(Pleasespecify)

[Text]

Links:

Englishversion:https://goo.gl/forms/4HZE4J3vkcVvbWJi1

Frenchversion:https://goo.gl/forms/6G6DHUFRi9YpnPEq2

Italianversion:https://goo.gl/forms/hjYc96O55LIAzAUJ3


55

11 AnnexII–TestingscenariosResults

11.1 BasicCatalogueServices

ScenarioGroup1 BasicCatalogueServices

Pre-requisites BasicICTskillsUserbrowsestheARIADNEPortal

Step Action Expectedresult Conclusion Validation1.1 Catalogue

NavigateVisualisationoftheentirecataloguewithinARIADNEPortal

Theportalallowstoaccesstotheentirecataloguebysearchingwithanemptyvalue

Yes

1.2 CatalogueSearch

Textualresearchonthecatalogue InthehomepageoftheARIADNEPortalthereisasearchwindowthatallowstextualresearchonthecatalogue

Yes

1.3 CatalogueView Visualisationofsearchresults ThecatalogueisrepresentedontheARIADNEportalasamultipagedynamictable(queryonelasticsearch).EachrecordhasitsownwebpagederivedtoARIADNERegistry

Yes

1.4 SpatialNavigate Navigatethecatalogueonawebmappingpage

Thegeolocationofthecataloguerecordsisdisplayableandnavigableonaleafletwebmappingpage(baseopenstreetmap)

Yes

1.5 SpatialSearch Searchforaspecificelementonawebmappingpage

Theresearchisavailablebyusingpanandzoomonaleafletwebmappingpageandbypushingonasearchbutton

Yes

1.6 SpatialView As1.4 As1.4 Yes1.7 Timeline

NavigateNavigatethecatalogueonatimelinegraph

Therepresentationofthetemporalattributeonagraphisavailableonajavascriptpage

Yes

1.8 TimelineSearch Searchforaspecifictimespanonatimelinegraph

Theresearchisavailablebyusingthemousedragonthejavascriptgraphpage

Yes

1.9 TimelineView As1.7 As1.7 Yes

11.2 AdvancedCatalogueServices


56

ScenarioGroup2 AdvancedCatalogueServices

Pre-requisites BasicICTskillsBasicgeosearchskillsBasictimesearchskillsUserbrowsestheARIADNEPortalUseradvancedsearchontheARIADNEPortal

Step Action Expectedresult Conclusion Validation2.1 Catalogue

ParameteriseSearch

Specifictextualresearch(e.g.«roman») ThesearchwindowonthePortalhomepageallowsspecifictextualsearch,suggestingsimilartermsfromAATthesaurus

Yes

2.2 GeoParameteriseSearch

Specificgeographicresearch(e.g.bycoordinatesorbyplacename)

Itisn’tpossibletoinsertaspecificcoupleofcoordinatesortoinsertaplacenamefortheresearch.Thegeographicsearchisonlyavailablebyusingpanandzoom(mouseclickandmousescroll)

No

2.3 TimelineParameterisesearch

Specifictimespanresearch(e.g.byperiodnameorspecificdate)

Itisn’tpossibletoinsertaspecificdateortoinsertaperiodnamefortheresearch.Thetimelinesearchisonlyavailablebyusingmousedrag

No

11.3 VisualMediaServices

ScenarioGroup3 VisualMediaServices

Pre-requisites AdvancedimageprocessingskillsUserhasavisualmediadocument(3Dmodel,RTI,HRimage)

Step Action Expectedresult Conclusion Validation3.1 3DModels

NavigateVisualisationofthepublicregistryof3DmodelsuploadedontheVisualMediaService

Thenavigationoftheitemsisavailablefromthehomepageoftheservice(button«browse»).Itispossiblealsotoaddatypefilter

Yes

3.2 3DModelsView Visualisationofeach3DmodelpublishedontheVisualMediaService

Fromthebrowsepageitispossibletoclickoneachitemtovisualisethemodelona3DHOPpage

Yes

3.3 3DModelsUpload

Uploadtheirown3DmodelsontheVisualMediaServiceondifferentformats

Theuploadofthemodelsisavailablefromthehomepageoftheservice(button«upload»).Sofar,theonlyformatacceptedisthe.ply

Yes

3.4 3DModelsParameter

Changetheparameterof3Dmodelmanagement

Theserviceallowsitemmetadatatobemodifiedandtosetupthevisualisationwebpageof3Dmodel

Yes

3.5 3DModelsTransform

3Dmodeltransformationformat Theserviceallowstransformationofthe.plymodelsinadownloadablewebpagebasedon3DHOP(.zip).Nomoretransformationisavailable

Yes

3.6 3DModelsDownload

3Dmodelsdownload Thedownloadingofthemodelsisavailableonlyfortheirownuploadedmodels.Itisn’tpossibletodownload

No


57

themodelsonthepublicgallery3.7 3DModels

Embed(onwebsite)

Embedding3Dmodelwebpageonitsownwebsite

The3DHOPwebpagecanbeembeddedonawebsiteasan«iframe»

Yes

3.8 RTINavigate VisualisationofthepublicregistryofRTIimagesuploadedontheVisualMediaService


Yes

3.9 RTIView VisualisationofeachRTIimagepublishedontheVisualMediaService

Fromthebrowsepageitispossibletoclickoneachitemtovisualizetheimageonawebpage

Yes

3.10 RTIUpload UploadtheirownRTIimagesontheVisualMediaServiceondifferentformats

TheuploadoftheRTIimagesisavailablefromthehomepageoftheservice(button«upload»).Theformatsacceptedare:.ptmand.hsh

Yes

3.11 RTIParameter ChangetheparameterofRTIimagemanagement

Theserviceonlyallowstomodifyitemmetadata

Yes

3.12 RTITransform RTIimagetransformationformat Theserviceallowsthetransformationof.ptmand.hshimagesinadownloadablewebpagebasedon3DHOP(.zip).Nomoretransformationisavailable

Yes

3.13 RTIDownload RTIimagesdownload ThedownloadingoftheRTIimagesisavailableonlyfortheirownuploadedfiles.Itisn’tpossibletodownloadthemodelsonthepublicgallery

No

3.14 RTIEmbed(onwebsite)

EmbeddingRTIimagewebpageonitsownwebsite


Yes

3.15 HighResImagesNavigate

VisualisationofthepublicregistryofHRimagesuploadedontheVisualMediaService


Yes

3.16 HighResImagesView

VisualisationofeachHRimagepublishedontheVisualMediaService

Fromthebrowsepageitispossibletoclickoneachitemtovisualizetheimageonawebpage

Yes

3.17 HighResImagesUpload

UploadtheirownHRimagesontheVisualMediaServiceondifferentformats

TheuploadoftheRTIimagesisavailablefromthehomepageoftheservice(button«upload»).Themostcommonimageformatsareaccepted

Yes

3.18 HighResImagesParameter

ChangetheparameterofHRimagemanagement

Theserviceonlyallowstomodifyitemmetadata

Yes

3.19 HighResImagesTransform

HRimagetransformationformat Theserviceallowsimagestobetransformed(indifferentformats)inadownloadablewebpagebasedon3DHOP(.zip).Nomoretransformationisavailable

Yes

3.20 HighResImagesDownload

HRimagesdownload ThedownloadingoftheHRimagesisavailableonlyfortheirownuploadedfiles.Itisn’tpossibletodownloadthemodelsonthepublicgallery

No

3.21 HighResImagesEmbed(onwebsite)

EmbeddingHRimagewebpageonitsownwebsite


Yes


58

11.4 OwnCloudServices

ScenarioGroup4 OwnCloudServices

Pre-requisites AdvancedICTskillsUserhassomegeographicaldatasets(Raster+DEM+shpfile)

Step Action Expectedresult Conclusion Validation4.1 OwnCloud

NavigateFilesandFoldersvisualisationonawebcloudstorage

Theserviceallowsnavigationtofoldersandfilesunauserfriendlywebcloudstorage

Yes

4.2 OwnCloud(Accessand)View

Privateaccesstothewebcloudstorage Theserviceasksforausernameandapasswordfortheprivateaccesstoitsownwebstorage

Yes

4.3 OwnCloudUpload

Uploadfilesonawebcloudstorage Theserviceallowsuploadingofdifferentsizefilesinauserfriendlywebcloudstorage(draganddropandbutton“upload”)

Yes

4.4 OwnCloudManage

Managethefoldersandthefilesonawebcloudstorage

Theserviceallowsuserstomanage,rename,delete,copy,cut,paste,etc.bothfoldersandfilesinauserfriendlywebcloudstorage

Yes

11.5 LandscapeServices

ScenarioGroup5 LandscapeServices

Pre-requisites BasicGISskillsUserhasuploadedageographicaldatasetonOwnCloud

Step Action Expectedresult Conclusion Validation5.1 3DTerrain

Navigate3Dterrainmodelsnavigationonawebgallery

The3DterraingalleryisavailablefromtheLandscapeservicehomepage

Yes

5.2 3DTerrainView 3Dterrainmodelvisualisation Fromthepublicgallery,eachmodelisaccessiblefortheonlinevisualisation(bothfromdesktopandmobile)

Yes

5.3 3DTerrainParameter

Changetheparameterof3Dterrainmodelmanagement

The3Dterrainserviceallowsuserstomodifymodelmetadata,modelparameter(format,resolution,layers,etc.)

Yes

5.4 3DTerrainUpload

Uploadtheirown3DterrainmodelsontheLandscapeServiceondifferentformats

Theserviceallowstouploaditemsondifferentfileformatsbyusing“OwnCloud”(see4.1-4.4)

Yes

5.5 3DTerrainTransform

3Dterrainmodeltransformationformat

The3Dterrainserviceallowsuserstotransformtheirown3Dterrainmodelsondifferentformats(e.g.JSONWebGL,3DS,OBJ,FBX,etc.)

Yes

5.6 TerrainGallery As5.1 As5.1 Yes


59

Navigate5.7 TerrainGallery

ViewAs5.1 As5.1 Yes

5.8 3DTerrainDownload

3Dterraindownload Thedownloadisavailableindifferentformatsforeachitempublishedonthepublicgalleryandfromitsownprivategallery(privateaccessvia“OwnCloud”)

Yes

5.9 3DTerrainEmbed(onwebsite)

Embedding3DTerrainwebpageonitsownwebsite

ThewebpagecontainingthemodelonJSONWebGLformatcanbeembeddedonawebsiteasan«iframe»

Yes

Data & Analytics

ARIADNE: Final testing report