Tool-supportedCollaborativeRequirementsPrioritisationP.Busetta,F.M.Kifetew,D.Muñante,A.Perini,A.Siena,A.Susi
6July2017COMPSAC2017
MotivationsandKeyPoints ataglance:• Decidingwhatismoreandwhatislessimportantiscriticalwhendevelopingsoftware,inparticulartodeterminethecontentsofadevelopmentcycle,aswellasinanyactivitywithlimitedtimeandresources• However,differentstakeholders(developers,users,operations,marketing…)mayhavedifferentperspectivesandsometimesconflictingopinionsontherelativepriorityofrequirementstobeaddressed
• Supportingdecision-makingtechniques/toolsaboundbutpracticaluseisrelativelyrare• … especiallyinsmallanddistributedorganizations,becauseofperceivedcostsandeffortsw.r.t.benefits
• Theprocessweproposeanditsaccompanyingtooltackletheseissuesby:• Minimizingthecostofset-upandexecutionofadecision-makingsession• Allowingittobeeasily repeated,withthesameordifferentparticipants/criteria/items• Distributingitintimeandspace(i.e.,notrequiringeverybodytobeatthesametimeinthesameplace)
• Insertinggamification elementstomakedecision-makingorganizationallyflexible,engagingfortheparticipants,andpotentiallyitselfsourceoffurtherorganizationalknowledge
• Experimentedinafewcasestudiessofar,theveryfirstbeingillustratedhere
06July2017 Tool-SupportedRequirementPrioritisation 2
Outline
• Context:Involvingtheend-usersdirectlyintheloopandfosteringcollaborationwithinthedevelopmentteamandwithstakeholders• Introducinggamificationintheprocess• Anexperiment:configurationandconclusions• CurrentstateandFuturework
06July2017 Tool-SupportedRequirementPrioritisation 3
ProjectContext:End-userfeedbackandmonitoringdataasdriversofsoftwareevolutionandadaptation
06July2017 Tool-SupportedRequirementPrioritisation 4
Usersarefirst-classactorswithin theloop–particularlyrelevantwiththeincreasingadoptionofCI/CDpipelines
Prioritizationisacriticalsteptoinformreleaseplanning.Criteriamaybemultiple;thus,thisactivitycanbeseenasaformofmulti-objectiveoptimizations
ResearchperformedaspartoftheH2020SUPERSEDEproject
(www.supersede.eu)
Usersfeedbackanddatafrommonitorsbecomerequirements tobeaddressedbythenextsoftwarerelease
Prioritizinginacollaborativeenvironment• Collaborationisrequiredtocapturedifferentperspectivesonaproduct,aswellasdifferentcompetencesandexperiencewithinateam• Theprocess(anditsaccompanyingtool)shouldallowtheinvolvementofdecision-makersthat donotcoveranytraditional,organization-determinedrole• Whichmayincludemorethanthesole“productowner”(inSCRUMterminology)• Thisisevenmoreimportantifend-users“sitatthetable”,notjustbyprovidingfeedbackbut,indeed,participatingtothedecision-makingprocess
• Ontheotherside,howtokeepparticipantsengaged,willingtoparticipate?• Especiallynon-developersorothersthatdonothaveaspecificdutyintheorganization
06July2017 Tool-SupportedRequirementPrioritisation 5
Gamifyingtheprocess(1/2)”Gamification”simplymeans“introducingelementstypicalofgames”forvariouspurposes,typicallytomakeaprocesssmootherandmoreengaging.Wetookafewtoaddresstherequirementsidentifiedbefore:• Multipleroleswithdifferentallowedactions.Theyareunrelated toanyorganizationalstructure,justlocaltoaspecific(game)session• Identifiedsofararesupervisor,opinionprovider,negotiator,thelatterintroducingawholeworldof(potentiallyrich)interactionsamongplayersthatgoesbeyondthedeterministicresultsofatechniqueoralgorithm
• Anon-linearworkflow,wheremovingfromonephasetoanotherisdeterminedeitherby(game)logicorbyaspecificrolewhile,withineachphase,allrolesinvolvedcanplaytheiractionsinanarbitraryorderandpossiblymultipletimes• Identifiedsofarare(1)set-up,(2)opinionprovisionandnegotiation,(3)closure.Set-upandclosurearecurrentlyforthesupervisoronly(butcouldbeautomatede.g.withreasoningaboutthestateofthegame),whilethemiddlephaseisforallotherplayers
06July2017 Tool-SupportedRequirementPrioritisation 6
Gamifyingtheprocess(2/2)• Playfulness (howmuchoneis“entertained”bythesystem),tobeintroducedinvariousways.Weexperimentedwith:• Competition andReward bymeansofpoints-ification (i.e.collectionofpointsaccordingtotheactionsperformedandtheirmodalityofexecution,inturnleadingtotheintroductionofleaderboards,badgesandsoon)
• Timepressure isboththeresultsofpoints-ification (thefasteroneplays,themorepointsonecollects)andbythebarepsychologicaltrickofshowingaclockticking
• Lightweight-nessofgameplay(i.e.thesimplicityandspeedatwhichplayerscanperformtheiractions),notonlytoimproveplayfulnessbuttosupportrepeatability
• Easyrepeatability,enablingmultiplerunswiththesameordifferentstakeholders(participants),thesameordifferentsetsofrequirements,thesameordifferentsetsofprioritizationcriteria,andsoon• Further,theintroductionofsomeformofKPIs(e.g.,pointscouldbecomputedaccordingtovariouscriteriaandkeptseparated)maybeexploitedtoletcompetenceandengagementemergeovertime
06July2017 Tool-SupportedRequirementPrioritisation 7
GRP(GamifiedcollaborativeRequirementsPrioritisation):Experimentalset-up• TheexperimentedprocessisbasedonthewellknownAHP (AnalyticHierarchyProcess)
• Pairwisecomparisons:decision-makerexpresseswhichoneoftwoitems(requirementsinourcase)ispreferredaccordingtoaspecificcriterion(e.g.uservalue,developmentcomplexity,…)
• 9-valuescaleusedtoexpresspreferences,rangingfrom“AisgreatlypreferredtoB”to“indifferent”to“BisgreatlypreferredtoA”.
• Numberofcomparisons=½items2 *criteria• But,astopping criteriacanbeimposed,andsupervisormaydecidetostopanyway.Computationscanusepartial
data
• Allopinionproviderscanexpresstheirpreferencesonallrequirements.Negotiation isautomaticallyrequestedbythesystemwhentwoormoreplayershavestronglydifferentopinions(contrastingpreferencesexceedathreshold)• Resolutioncurrentlyimplementedasnegotiatorexpressingfinalopinionorignoringandlettingtheunderlyingcomputationsuseaverages
• ThetoolprovidesaWebinterface.Playerscanintervenefromanylocationattheirleisurebutwithinatimewindowdefinedbythesupervisor• Gamificationthusincludedmanyplayfulnessaspects,butnopoints-ification (competition&rewards)fortheexperimentreportedhere
06July2017 Tool-SupportedRequirementPrioritisation 8
Webinterfaces
• Gameset-upandconclusionwithcomputations(supervisor)• Pairwisepreferences(opinionproviders)• Conflictresolution(negotiator)
06July2017 Tool-SupportedRequirementPrioritisation 9
Pointsandbadges(NOTE:laterversion)
Preferenceexpression
Playerhistory
Theexperiment:dataandcharacteristics• DevelopmentteamofR&Ddepartment• DeltaInformatica’s owngroupworkingonauthoringtoolsforvirtualreality-basedtraining(PRESTOtoolsuite)
• SME.Littleornouserinputonmostnovelaspects• 16requirements,groupedin4clusters• 2criteriaforevaluation(effortanduservalue)• 8participants:engineersofvarioussenioritylevelsandtheircoordinator• Allasopinionproviders• Thecoordinatorhadalsotheroleofnegotiator
• 24hoursfor“playing”4games(onepercluster)• Questionnairesandinterviewsafterconclusion
06July 2017 Tool-SupportedRequirementPrioritisation 10
Thesubjectofprioritization:Averycontroversialuserinterface
PRESTOScripting’scontrolleristhekeytoolforatrainertostartanddirectatrainingsession
inavirtualreality
(seediscussioninP.Busetta,AddressingTeamAwarenessByMeansOfARequirementPrioritizationTool,PRIORE’17)
Resultsfromtheexperiment• GRPperceivedaseffective forreachingacceptabledecisionsinanefficientmanner• Participantsagreedtoahighdegreewiththefinaloutcomeoftheprocessforallclustersofrequirements• Thisinspiteofahighlevelofdisagreementinpairwisepreferences• ThecomparisonsofGRPoutcomesbeforeandafterthenegotiator’sinterventionshowedthatthenegotiatorplaysanimportantrole inmanagingconflictingpreferences,affectingresults
• Reportedadvantagesincludetheabilitytotrackdecisions• Suggestedpotentialimprovementsincludemechanismsfor“livecommunication”duringgameplay• Themeasuredtimerequiredtoexpresspreferencesisquiteshort
06July2017 Tool-SupportedRequirementPrioritisation 12
Currentstate
• Improvedprototypeandextendedscopeofprocess• Points-ifcation fullysupported• PrioritizationtechniquesalternativetoAHPhavebeenintroduced,mostimportantlynon-pairwisecomparisons(usingGAstocomputeagreements,measuringindividualchoice’sdistancefromgroup’sconsensus,…)
• Revisedtheentireprocesstogivemoreimportancetothesupervisorandtointroduceplanningtools,e.g.toselectthemostsuitablealgorithm/technique,todecidehowmanygamestoplayandwithwhichparticipants,andsoon
• Triedoutinotherexperimentsandvalidatedagainsttheproject’susecases• Resultsavailableasproject’sreports• Overall,approachalwayswellreceived
06July2017 Tool-SupportedRequirementPrioritisation 13
Futuredirections
• Integrationofprioritizationprocesswithsomeexistingapproach/methodology• LookinginparticularatSCRUMandatitsSprints’“backlog”selectionand“planning”meeting
• Integrationofprioritizationtoolasplug-inofmarkettools• Currently,developingplug-insforAtlassian’s JIRA,extendingitsAgiletools
06July2017 Tool-SupportedRequirementPrioritisation 14
06July2017 Tool-SupportedRequirementPrioritisation 15
Thanksforyourattention!