ExtendingExistingInferenceToolstoMineDynamicAPIs

ZiyadAlsaeedandMichalYoungUniversityofOregon

June2,20182nd InternationalWorkshoponAPIUsageandEvolution(WAPI)

Motivation

• APIunderstandingisakeytosolvemanysoftwareusageissues.

• Softwaredocumentationsarerarelyup-to-dateandconstraintsassociatedwithobjectsareusuallyinthebrainofthecreator.

• Howdowecapturethesoftwaredynamicnature.

26/2/2018 WAPI'18- Z.Alsaeed&M.Young

ExistingDynamicInferenceTools(Daikon)

6/2/2018 3

extremelytiedtoestablished

TheMajorityofourcontribution

Chicory Daikon

[1]Ernstet.al.ICSE’99

WAPI'18- Z.Alsaeed&M.Young

MiningtheWell-KnownStackAr byDaikon

6/2/2018 4WAPI'18- Z.Alsaeed&M.Young

HowChicoryWorks

6/2/2018 5WAPI'18- Z.Alsaeed&M.Young

HowChicoryWorks

x:::EXIT 𝑓"# 𝑓"$# …𝑎&##

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0

6/2/2018 6WAPI'18- Z.Alsaeed&M.Young

HowChicoryWorks

x:::EXIT 𝑓"# 𝑓"$#…

𝑎&##𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- -1 null 0.0

6/2/2018 7WAPI'18- Z.Alsaeed&M.Young

HowChicoryWorks

x:::EXIT 𝑓"# 𝑓"$#…

𝑎&##𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- -1 null 0.0

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛. -2 null 0.0

6/2/2018 8WAPI'18- Z.Alsaeed&M.Young

HowChicoryWorks

x:::EXIT 𝑓"# 𝑓"$#…

𝑎&##𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- -1 null 0.0

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛. -2 null 0.0

6/2/2018 9

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛& 0 null 0.0

… …WAPI'18- Z.Alsaeed&M.Young

Whatif?

6/2/2018 10WAPI'18- Z.Alsaeed&M.Young

SimplifiedReal-WorldExample

6/2/2018 11WAPI'18- Z.Alsaeed&M.Young

SimplifiedReal-WorldExample

6/2/2018 12WAPI'18- Z.Alsaeed&M.Young

VariablesStructureofmodify basedonChicory:

6/2/2018 13WAPI'18- Z.Alsaeed&M.Young

PossibleScenariosWhenObservingaDynamicDataStructure

• Elementintroduction(addedtoDDS):• AnelementthatwasnotpresentintheDDSuntilalaterpointintheprogramexecutionandneverremovedthereafter.

• Elementremoval(removedfromDDS):• Anelementthatexistsatsomepointoftheprogramexecutionbutremovedbeforethelastobservationoftheprogrampoint.

• Andmore…

6/2/2018 14WAPI'18- Z.Alsaeed&M.Young

PossibleStructureEvolution

6/2/2018 15WAPI'18- Z.Alsaeed&M.Young

RelationtoDaikon

• Daikonexpectsaverywelldefinedstructureofaprogrampoint(methodentranceorexit).

• Onlyonevariablestructuretreeperprogrampoint.

6/2/2018 16WAPI'18- Z.Alsaeed&M.Young

eChicory Structure

6/2/2018 17WAPI'18- Z.Alsaeed&M.Young

UnificationPhaseineChicory (elementintroduction)

6/2/2018 18

0 0 01 2 1

WAPI'18- Z.Alsaeed&M.Young

UnificationPhaseineChicory (elementintroduction)

6/2/2018 19

modify:::ENTER 𝑟". 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙𝑉𝑎𝑙𝑢𝑒 𝑟-. 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙𝑉𝑎𝑙𝑢𝑒 𝑟.. 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙𝑉𝑎𝑙𝑢𝑒

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 0 nonsensical nonsensical

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- 1 0 nonsensical

𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛. 2 1 0

WAPI'18- Z.Alsaeed&M.Young

UnificationPhaseineChicory (elementremoval)

• GivenDaikondesign,removingavariablecanonlybeachievedbymanipulatingitstrace.

• Changingatracetononsensicalafteritwasinitializedisprohibited.

• Givenitarbitraryvalue(e.g.stretchitslastknownvaluetotherestofinstanceswhereitwasremovedthereafter)wouldinterferewiththeinvariantsintegrity.

6/2/2018 20WAPI'18- Z.Alsaeed&M.Young

Evaluation- ArtifactSelectionCriteria

• SourceavailableinGitHub.

• Applicationsofsizebetween2Kand10KLOC.

• Hasindicationsthatoneoftheselectedpatternsisused(giventherepositoryissuestracker,pullrequests,andwiki).

• Hashightestcoverage(ifreported).

• Popularorwellmaintainedapplicationsbasedonthestarrateormanagingorganization.

6/2/2018 21WAPI'18- Z.Alsaeed&M.Young

Evaluation- SelectedArtifacts

Application Description Selected Classes #ofMethods

Represented DesignPattern

Mockito MockingframeworkforunittestsinJava

InvocationNotierHandler 7 ObserverPattern

ApacheStruts FrameworkforcreatingJava webapplications

DefaultActionInvocation 29

MVC

DefaultUnknownHandlerManager 3

CongurationManager 16

VelocityManager 18

SimpleTextNode 17

SimpleAdapterDocument 43

JabRef BibTeX Managementapplication

EntryEditor 22

MVCCleanupActionsListModel 8

UndoableModifySubtree 4

ImportInspectionDialog 21

Zeppelin Awebbasedinteractivedataanalytictool

Folder 23

ObserverPatternNotebook 45

NotebookRepoSync 31

6/2/2018 22WAPI'18- Z.Alsaeed&M.Young

EvaluationCriteria(PrecisionandRecall)

• Pros:• Hasbeingthebaseforevaluatingspecificationminers.

• Showsagoodinsightabouttheaccuracyofthespecificationminer.

• Cons:• Agroundtruthaboutthetestsubjectmustbedefinedahead(thisisdonebyhumans,thuscan’tbescaled).

• Humandefinedgroundtruth,candifferedbasedonthedevelopersvieworopinion.

6/2/2018 23WAPI'18- Z.Alsaeed&M.Young

EvaluationCriteria(PurityAnalysis)

• Thenotionofpure(side-effectfree)methodsiswell-definedinthestaticanalysisdomain.

• Canbegeneratedautomaticallyandscalewithlargeapplications.

• Notthegoalofdynamicanalysis,butcanbeusedtocheckconsistency.

6/2/2018 24WAPI'18- Z.Alsaeed&M.Young

Mockito - InvocationNotifierHandler

6/2/2018 25

Method jPure eChicory ChicoryInvocationNotifierHandler(InternalMockHandler<T>, MockCreationSettings<T>) !pure !pure !purehandle(Invocation) !pure !pure purenotifyMethodCall(Invocation, Object) !pure !pure purenotifyMethodCallException(Invocation, Throwable) !pure !pure puregetMockSettings() !pure pure puregetInvocationContainer() !pure pure puresetAnswersForStubbing(List<Answer<?>>) !pure pure pure

Totalnumberofreportedmethodswith noindicationofeffect 0 3 6

WAPI'18- Z.Alsaeed&M.Young

Mockito - InvocationNotifierHandlerChicory eChicory

6/2/2018 26WAPI'18- Z.Alsaeed&M.Young

ImpedimentstoObserveOtherSelectedTestSubjects• Inadequateinputs(unittests).

• Naiveimplementationofconcreteclasses.

• AbsenceofelementsinDDS.

• TestsFailures(confirmedbyrepositorymaintainers).

6/2/2018 27WAPI'18- Z.Alsaeed&M.Young

WroteComprehensiveTeststoProveourApproachPotential• Toprovetheprovidedtestsarethecauseofresultlimitations,wewroteunitetestsforoneoftheclassesfromApacheStruts.

• ClassDefaultUnknownHandlerManager• Method-1:handleUnknownAction• Method-2:handleUnknownMethod

• WrittentestsarereviewedandmergedintoApacheStruts’mainrepository.

6/2/2018 WAPI'18- Z.Alsaeed&M.Young 28

Performance- TracesCollectionPhase

6/2/2018 29WAPI'18- Z.Alsaeed&M.Young

Performance- InferencePhase

5.32

3.72

1.50 2.90 3.31

41.62

12.21

5.096.40

3.30

1.59 2.79 3.83

13.24

8.69

3.90

TIME(SEC

ONDS

)

eChicory Chicory

6/2/2018 30WAPI'18- Z.Alsaeed&M.Young

Conclusion

• Wehighlightedthenon-fullydynamictracingissueandclearlyidentifiedthatlimitationsofcurrentinstrumentationmethodologies.

• Weimplementedaprototypeasaproofofconcepttofullydynamicallyobservecomplexsystems.

• Weshowedbyrealworldexamplethatexistinginstrumentationtechniquesareblindtocommondesignpatternsare.

• DDSsareonlyonesourceofprogramstructuralchange.Therearedifferentprogrammingpracticesthatleadstoverydynamicstructureneedstobeaddressed.

6/2/2018 31WAPI'18- Z.Alsaeed&M.Young

Thankyou.

6/2/2018 32

References:

[1]Ernst,M.D.;Cockrell,J.;Griswold,W.G.&Notkin,D.DynamicallyDiscoveringLikelyProgramInvariantstoSupportProgramEvolutionProceedingsofthe21stInternationalConferenceonSoftwareEngineering,ACM,1999,213-224

WAPI'18- Z.Alsaeed&M.Young