Explaining why methods change together

Explaining why methods change together

Angela Lozano, Carlos Noguera, Viviane Jonckers Vrije Universiteit Brussel, Belgium

1

Why co-changes?Reveal hidden dependencies [Gall, Hajek, Jazayeri ICSM 1998]

Identify restructuring candidates [Gall, Hajek, Jazayeri ICSM 1998, Girba, Ducasse, Lanza ICSM 2004]

Predict change propagation [Hassan & Holt ICSM 2004]

Validate the completeness of a change [Zimmermann, Weisgerber, Diehl, Zeller ICSE 2004]

Support some change tasks [Robillard, Dagenais JSME 2010]

2

Explaining co-changes

Find out the reason of the co-change

• Out-perform co-changes

• Useful for impact analysis of new entities

!

3

Explaining co-changes

Find out the reason of the co-change

• Out-perform co-changes

• Useful for impact analysis of new entities

!

Reason: common properties (structural/semantic) of co-changing methods

3

For instance*

CALLS_METHOD_NAME:goodsType, LOCAL_VARIABLE_DECLARATION_NAME:good, !METHOD_JAVADOC_MENTIONS:manufactured, METHOD_JAVADOC_MENTIONS:material, ! !METHOD_JAVADOC_MENTIONS:raw, METHOD_JAVADOC_MENTIONS:type

net.sf.freecol.common.model.Goods.getRawMaterial(int) Inet.sf.freecol.common.model.Goods.getManufactoredGoods(int) I

"Fixes a bug in getRawMaterial and getManufactoredGoods."

* The typos you see in the examples come from the data collected

4

Commit transaction

Cluster

Time (commit transactions)

Ent

ities

(met

hods

) m1 Im2 I I I I I I Im3 I I I I I I Im4 I I I I I I Im5 I I I I I I Im6 I I I I I I I Im7 I I I I I I I I Im8 I I I I I Im9 I I I I I I Im0 I I

Identifying co-change relations

5

…

!!RETURN_TYPE: !

METHOD_PARAM_TYPE: !DECLARING_TYPE: !

DECLARING_TYPE_EXTENDS: !

DECLARING_TYPE_IMPLEMENTS: !

LOCAL_VARIABLE_DECLARATION_TYPE:

Structural properties (a.k.a. Syntactic / Explicit)

6

…

Semantic properties (a.k.a. Lexical / Implicit)

!!METHOD_NAME: !

CALLS_METHOD_NAME: !

LOCAL_VARIABLE_DECLARATION_NAME: !

METHOD_PARAM_NAME: !

METHOD_JAVADOC_MENTIONS: !

7

Are these good reasons?

8

Coverage: Describes most co-changes !

!

!

Idiosyncrasy: Describes only co-changing methods

What is a good reason?

defined above, we describe the methods on this committransaction using 93 properties (50 for the first one,and 43 for the second one). Since the methods shared 9characteristics, the reason for the commit transaction becomes:CALLS_METHOD_NAME: add, equals, get, getLength, size,toString; LOCAL_VARIABLE_DECLARATION_NAME:i, task; and LOCAL_VARIABLE_DECLARATION_TYPE:GanttTask.

B. Some reasons are better than others

With this set of properties as possible explanations for co-changes, we aim at answering five research questions:

RQ1: To what extent is it possible to automatically find areason for a set of co-changing methods? We analyze thisquestion by comparing the coverage of the reasons found:

Coverage: This relates to the number of commit-transactions that have non-empty reasons. We define two typesof coverage.

Coverage per commit, CovC as the ratio of commits witha non-empty CR reason to total number of commits in thesystem’s history Cs. And coverage per methods, CovM as theratio of methods with a non-empty MR reason† to total numberof methods in the system’s history Ms.

CovC = CRCs

, CovM = MRMs

Given that we eliminate commits in which only one methodchanges, and that commits in which many methods change areunlikely to have a single reason, the coverage of our approachwill be low.

RQ2: To what extent the automatically detected reasonsdescribe only the set of co-changing methods? We analyzethis question by assessing the discriminating power of reasons.

Idiosyncrasy: Good reasons will contain properties thattend to occur only in methods that change together. If theproperties found in a reason are also found in methods thatdid not change together, then those properties are likely foundby a coincidence and do not represent an explanation for thechange.

Therefore, we measure the idiosyncrasy Idios(RM ) of areason RM as one minus the ratio between the set of methodsthat are described by RM by coincidence (i.e., the number ofmethods that have properties in common with RM but thatdo not belong to the methods it describes –M–) and the totalnumber of methods in the system’s history Ms.

Idios(RM ) = 1− | ∪m∈Ms RM ⊂ Dm|− |M ||Ms|

For example the idiosyncrasy for the example commit is: 1-((5‡ - 2§)/(14895¶) ) = 1- 0.0002 = 0.9998.

†Methods modified in at least one commit with non-empty reasons.‡Methods whose description includes all the properties of the example

commit: GanttXMLSaver.writeTask(OutputStreamWriter,int,String)GanttXMLOpen.DefaultTagHandler.startElement(String,String,String,Attributes)GanttXMLOpen.GanttXMLParser.startElement(String,String,String,Attributes)GanttXMLSaver.writeTask(OutputStreamWriter,DefaultMutableTreeNode,String)GanttXMLSaver.writeTask(Writer,DefaultMutableTreeNode,String)

§Number of methods modified in the example commit see section II-A.¶Methods belonging to GanttProject during the period analyzed (table III)

RQ3: To what extent the automatically detected reasons forco-changing methods overlap with each other? We analyzethis question by measuring the uniqueness of reasons.

Uniqueness: It is also important to know whether rea-sons are sufficiently different between each other to serveas explanations only for the changes they describe. Thus, wemeasure the similarity Sim(R1, R2) between two reasons astheir Jaccard index (i.e., the intersection over the union of theirproperties.).

The uniqueness of a reason Ri is the mean difference tothe rest of reasons found in the project (i.e., R).

Unq(Ri) = 1− x̃(!

Rj∈R∧i ̸=j

Sim(Ri, Rj))

Uniqueness tells us if different co-change relations are likelyto be due to different reasons. Lets suppose that there are threemethods (m1, m2, and m3) but there are only two co-changeclusters (m1 and m2, m2 and m3). Even though m2 co-changeswith m1 and m3, it is likely that the reasons for co-changingwith m2, are different from the reasons for co-changing withm3. Therefore we expect the reasons to be unique. For examplethe uniqueness for the example commit is∥ 0.985333.

RQ4: To what extent the automatically detected reasonsfor sets of co-changing methods are sound? We analyzethis question by manually checking their plausibility.

Plausibility: Even if reasons explain a large number ofmethods (high CovM ), covering most of the history of theapplication (high CovC), being sufficiently discriminatory(high Idios, and high Unq), they might still make no sense.Therefore, we need to perform a manual assessment of thereasons found to see whether they are plausible. This isachieved by comparing the reasons for a (set) of commitsproduced by our analysis with the message that corresponds tothe (set of) commits. We assert each reason as either plausibleor not. We consider a reason a plausible explanation for acommit, if the words or terms mentioned in the properties ofthe reason appear in the commit message that accompanies theco-change. For groups of co-changes, for those that span morethan 6 co-changes, we extract a word-cloud from the commitmessages to provide an overview of the general terms therein.We apply a small degree of liberty when aligning the termsof the reason with those present in the commit message(s);for example if the commit message mentions UI, or mouseevents, we will consider plausible reasons those that includedependencies to JPanel or ActionListener types (both typespresent in Java’s SWING UI framework). Finally, note that theplausibility depends on the quality of the commit message, ifthe commit message provides no information (i.e., “bugfix” or“no message”) we will consider the reason as an implausibleexplanation for the commit regardless of the properties foundin the reason itself.

The example commit is considered plausible because thecommit message and the reason refer to an I/O task∗∗.

∥The example commit had a non-empty intersection with 128 commits(from the 626 commits in GanttProject with a non-empty reason). Moreover,most of the intersections had only one property. Therefore, its similarity withother commits is very low (Min: 0, 1st Qu.:0, Median:0, Mean:0.018, 3rdQu.:0, Max:0.3).

∗∗See the description of the example in section II-A.

defined above, we describe the methods on this committransaction using 93 properties (50 for the first one,and 43 for the second one). Since the methods shared 9characteristics, the reason for the commit transaction becomes:CALLS_METHOD_NAME: add, equals, get, getLength, size,toString; LOCAL_VARIABLE_DECLARATION_NAME:i, task; and LOCAL_VARIABLE_DECLARATION_TYPE:GanttTask.

B. Some reasons are better than others

With this set of properties as possible explanations for co-changes, we aim at answering five research questions:

RQ1: To what extent is it possible to automatically find areason for a set of co-changing methods? We analyze thisquestion by comparing the coverage of the reasons found:

Coverage: This relates to the number of commit-transactions that have non-empty reasons. We define two typesof coverage.

Coverage per commit, CovC as the ratio of commits witha non-empty CR reason to total number of commits in thesystem’s history Cs. And coverage per methods, CovM as theratio of methods with a non-empty MR reason† to total numberof methods in the system’s history Ms.

CovC = CRCs

, CovM = MRMs

Given that we eliminate commits in which only one methodchanges, and that commits in which many methods change areunlikely to have a single reason, the coverage of our approachwill be low.

RQ2: To what extent the automatically detected reasonsdescribe only the set of co-changing methods? We analyzethis question by assessing the discriminating power of reasons.

Idiosyncrasy: Good reasons will contain properties thattend to occur only in methods that change together. If theproperties found in a reason are also found in methods thatdid not change together, then those properties are likely foundby a coincidence and do not represent an explanation for thechange.

Therefore, we measure the idiosyncrasy Idios(RM ) of areason RM as one minus the ratio between the set of methodsthat are described by RM by coincidence (i.e., the number ofmethods that have properties in common with RM but thatdo not belong to the methods it describes –M–) and the totalnumber of methods in the system’s history Ms.

Idios(RM ) = 1− | ∪m∈Ms RM ⊂ Dm|− |M ||Ms|

For example the idiosyncrasy for the example commit is: 1-((5‡ - 2§)/(14895¶) ) = 1- 0.0002 = 0.9998.

†Methods modified in at least one commit with non-empty reasons.‡Methods whose description includes all the properties of the example

commit: GanttXMLSaver.writeTask(OutputStreamWriter,int,String)GanttXMLOpen.DefaultTagHandler.startElement(String,String,String,Attributes)GanttXMLOpen.GanttXMLParser.startElement(String,String,String,Attributes)GanttXMLSaver.writeTask(OutputStreamWriter,DefaultMutableTreeNode,String)GanttXMLSaver.writeTask(Writer,DefaultMutableTreeNode,String)

§Number of methods modified in the example commit see section II-A.¶Methods belonging to GanttProject during the period analyzed (table III)

RQ3: To what extent the automatically detected reasons forco-changing methods overlap with each other? We analyzethis question by measuring the uniqueness of reasons.

Uniqueness: It is also important to know whether rea-sons are sufficiently different between each other to serveas explanations only for the changes they describe. Thus, wemeasure the similarity Sim(R1, R2) between two reasons astheir Jaccard index (i.e., the intersection over the union of theirproperties.).

The uniqueness of a reason Ri is the mean difference tothe rest of reasons found in the project (i.e., R).

Unq(Ri) = 1− x̃(!

Rj∈R∧i ̸=j

Sim(Ri, Rj))

Uniqueness tells us if different co-change relations are likelyto be due to different reasons. Lets suppose that there are threemethods (m1, m2, and m3) but there are only two co-changeclusters (m1 and m2, m2 and m3). Even though m2 co-changeswith m1 and m3, it is likely that the reasons for co-changingwith m2, are different from the reasons for co-changing withm3. Therefore we expect the reasons to be unique. For examplethe uniqueness for the example commit is∥ 0.985333.

RQ4: To what extent the automatically detected reasonsfor sets of co-changing methods are sound? We analyzethis question by manually checking their plausibility.

Plausibility: Even if reasons explain a large number ofmethods (high CovM ), covering most of the history of theapplication (high CovC), being sufficiently discriminatory(high Idios, and high Unq), they might still make no sense.Therefore, we need to perform a manual assessment of thereasons found to see whether they are plausible. This isachieved by comparing the reasons for a (set) of commitsproduced by our analysis with the message that corresponds tothe (set of) commits. We assert each reason as either plausibleor not. We consider a reason a plausible explanation for acommit, if the words or terms mentioned in the properties ofthe reason appear in the commit message that accompanies theco-change. For groups of co-changes, for those that span morethan 6 co-changes, we extract a word-cloud from the commitmessages to provide an overview of the general terms therein.We apply a small degree of liberty when aligning the termsof the reason with those present in the commit message(s);for example if the commit message mentions UI, or mouseevents, we will consider plausible reasons those that includedependencies to JPanel or ActionListener types (both typespresent in Java’s SWING UI framework). Finally, note that theplausibility depends on the quality of the commit message, ifthe commit message provides no information (i.e., “bugfix” or“no message”) we will consider the reason as an implausibleexplanation for the commit regardless of the properties foundin the reason itself.

The example commit is considered plausible because thecommit message and the reason refer to an I/O task∗∗.

∥The example commit had a non-empty intersection with 128 commits(from the 626 commits in GanttProject with a non-empty reason). Moreover,most of the intersections had only one property. Therefore, its similarity withother commits is very low (Min: 0, 1st Qu.:0, Median:0, Mean:0.018, 3rdQu.:0, Max:0.3).

∗∗See the description of the example in section II-A.

9

Uniqueness: Differs from other reasons !

!

!

Plausibility: Makes sense !

What is a good reason?

defined above, we describe the methods on this committransaction using 93 properties (50 for the first one,and 43 for the second one). Since the methods shared 9characteristics, the reason for the commit transaction becomes:CALLS_METHOD_NAME: add, equals, get, getLength, size,toString; LOCAL_VARIABLE_DECLARATION_NAME:i, task; and LOCAL_VARIABLE_DECLARATION_TYPE:GanttTask.

B. Some reasons are better than others

With this set of properties as possible explanations for co-changes, we aim at answering five research questions:

RQ1: To what extent is it possible to automatically find areason for a set of co-changing methods? We analyze thisquestion by comparing the coverage of the reasons found:

Coverage: This relates to the number of commit-transactions that have non-empty reasons. We define two typesof coverage.

Coverage per commit, CovC as the ratio of commits witha non-empty CR reason to total number of commits in thesystem’s history Cs. And coverage per methods, CovM as theratio of methods with a non-empty MR reason† to total numberof methods in the system’s history Ms.

CovC = CRCs

, CovM = MRMs

Given that we eliminate commits in which only one methodchanges, and that commits in which many methods change areunlikely to have a single reason, the coverage of our approachwill be low.

RQ2: To what extent the automatically detected reasonsdescribe only the set of co-changing methods? We analyzethis question by assessing the discriminating power of reasons.

Idiosyncrasy: Good reasons will contain properties thattend to occur only in methods that change together. If theproperties found in a reason are also found in methods thatdid not change together, then those properties are likely foundby a coincidence and do not represent an explanation for thechange.

Therefore, we measure the idiosyncrasy Idios(RM ) of areason RM as one minus the ratio between the set of methodsthat are described by RM by coincidence (i.e., the number ofmethods that have properties in common with RM but thatdo not belong to the methods it describes –M–) and the totalnumber of methods in the system’s history Ms.

Idios(RM ) = 1− | ∪m∈Ms RM ⊂ Dm|− |M ||Ms|

For example the idiosyncrasy for the example commit is: 1-((5‡ - 2§)/(14895¶) ) = 1- 0.0002 = 0.9998.

†Methods modified in at least one commit with non-empty reasons.‡Methods whose description includes all the properties of the example

commit: GanttXMLSaver.writeTask(OutputStreamWriter,int,String)GanttXMLOpen.DefaultTagHandler.startElement(String,String,String,Attributes)GanttXMLOpen.GanttXMLParser.startElement(String,String,String,Attributes)GanttXMLSaver.writeTask(OutputStreamWriter,DefaultMutableTreeNode,String)GanttXMLSaver.writeTask(Writer,DefaultMutableTreeNode,String)

§Number of methods modified in the example commit see section II-A.¶Methods belonging to GanttProject during the period analyzed (table III)

RQ3: To what extent the automatically detected reasons forco-changing methods overlap with each other? We analyzethis question by measuring the uniqueness of reasons.

Uniqueness: It is also important to know whether rea-sons are sufficiently different between each other to serveas explanations only for the changes they describe. Thus, wemeasure the similarity Sim(R1, R2) between two reasons astheir Jaccard index (i.e., the intersection over the union of theirproperties.).

The uniqueness of a reason Ri is the mean difference tothe rest of reasons found in the project (i.e., R).

Unq(Ri) = 1− x̃(!

Rj∈R∧i ̸=j

Sim(Ri, Rj))

Uniqueness tells us if different co-change relations are likelyto be due to different reasons. Lets suppose that there are threemethods (m1, m2, and m3) but there are only two co-changeclusters (m1 and m2, m2 and m3). Even though m2 co-changeswith m1 and m3, it is likely that the reasons for co-changingwith m2, are different from the reasons for co-changing withm3. Therefore we expect the reasons to be unique. For examplethe uniqueness for the example commit is∥ 0.985333.

RQ4: To what extent the automatically detected reasonsfor sets of co-changing methods are sound? We analyzethis question by manually checking their plausibility.

Plausibility: Even if reasons explain a large number ofmethods (high CovM ), covering most of the history of theapplication (high CovC), being sufficiently discriminatory(high Idios, and high Unq), they might still make no sense.Therefore, we need to perform a manual assessment of thereasons found to see whether they are plausible. This isachieved by comparing the reasons for a (set) of commitsproduced by our analysis with the message that corresponds tothe (set of) commits. We assert each reason as either plausibleor not. We consider a reason a plausible explanation for acommit, if the words or terms mentioned in the properties ofthe reason appear in the commit message that accompanies theco-change. For groups of co-changes, for those that span morethan 6 co-changes, we extract a word-cloud from the commitmessages to provide an overview of the general terms therein.We apply a small degree of liberty when aligning the termsof the reason with those present in the commit message(s);for example if the commit message mentions UI, or mouseevents, we will consider plausible reasons those that includedependencies to JPanel or ActionListener types (both typespresent in Java’s SWING UI framework). Finally, note that theplausibility depends on the quality of the commit message, ifthe commit message provides no information (i.e., “bugfix” or“no message”) we will consider the reason as an implausibleexplanation for the commit regardless of the properties foundin the reason itself.

The example commit is considered plausible because thecommit message and the reason refer to an I/O task∗∗.

∥The example commit had a non-empty intersection with 128 commits(from the 626 commits in GanttProject with a non-empty reason). Moreover,most of the intersections had only one property. Therefore, its similarity withother commits is very low (Min: 0, 1st Qu.:0, Median:0, Mean:0.018, 3rdQu.:0, Max:0.3).

∗∗See the description of the example in section II-A.

20% random sample manually compared to commit message

10

Empirical study

CVS repository

Freecol GanttProject

11

Empirical study

commit transactions

CVS repository

Freecol GanttProject

1.087 2.701

11

Empirical study

commit transactions properties

semantic structuralCVS repository

Freecol GanttProject

1.087 2.701

478.312 in 4.099 methods 547.394 in 14.895 methods

11

Empirical study

commit transactions properties

semantic structural

CmBoth CmSm CmSt

CVS repository

Freecol GanttProject

1.087 2.701

478.312 in 4.099 methods 547.394 in 14.895 methods

Reasons

11

Empirical study

commit transactions

clusters

propertiessemantic structural

CmBoth CmSm CmSt

CVS repository

Freecol GanttProject

1.087 2.701

14 280

478.312 in 4.099 methods 547.394 in 14.895 methods

Reasons

11

Empirical study

commit transactions

clusters

propertiessemantic structural

ClBoth ClSm ClSt

CmBoth CmSm CmSt

CVS repository

Freecol GanttProject

1.087 2.701

14 280

478.312 in 4.099 methods 547.394 in 14.895 methods

Reasons

Reasons

11

Results!

12

Are these good reasons?

Coverage (bad. lower for clusters.) Idiosyncrasy (good. lower for clusters.)

• Which properties are better? • Both prop. >> Structural prop. only • Both prop. > Semantic prop. only

13

Are these good reasons?

Coverage (bad. lower for clusters.) Idiosyncrasy (good. lower for clusters.)

• Which properties are better? • Both prop. >> Structural prop. only • Both prop. > Semantic prop. only

13

Are these good reasons?

The reasons tend to be unique (usual similarity < 5% & 10%, worst case <10% & 20%).

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.90 0.94 0.98Freecol

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.85 0.95GanttProject

UN

IQU

ENES

S

Uniqueness (good. cluster ? commit) !!!

14

Are these good reasons?

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.90 0.94 0.98Freecol

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.85 0.95GanttProject

UN

IQU

ENES

S

Uniqueness (good. cluster ? commit) !!!

14

Are these good reasons?

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.90 0.94 0.98Freecol

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.85 0.95GanttProject

UN

IQU

ENES

S

Uniqueness (good. cluster ? commit) !!!

14

!• Commits:

• Both prop. is better. Structural ≃ Semantic • Clusters:

• Semantic -> Both -> Structural

Are these good reasons?

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.90 0.94 0.98Freecol

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.85 0.95GanttProject

UN

IQU

ENES

S

Uniqueness (good. cluster ? commit) !!!

14

!• Commits:

• Both prop. is better. Structural ≃ Semantic • Clusters:

• Semantic -> Both -> Structural

!• Commits:

• Both prop. is better. Structural ≃ Semantic • Clusters:

• Semantic -> Both -> Structural

Are these good reasons?

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.90 0.94 0.98Freecol

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.85 0.95GanttProject

UN

IQU

ENES

S

Uniqueness (good. cluster ? commit) !!!

14

!• Commits:

• Both prop. is better. Structural ≃ Semantic • Clusters:

• Semantic -> Both -> Structural

!• Commits:

• Both prop. is better. Structural ≃ Semantic • Clusters:

• Semantic -> Both -> Structural

Are these good reasons?

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.90 0.94 0.98Freecol

clSm

clSt

clAll

cmSm

cmSt

cmAll

0.85 0.95GanttProject

UN

IQU

ENES

S

Uniqueness (good. cluster ? commit) !!!

14

!• Commits:

• Both prop. is better. Structural ≃ Semantic • Clusters:

• Semantic -> Both -> Structural

!• Commits:

• Both prop. is better. Structural ≃ Semantic • Clusters:

• Semantic -> Both -> Structural

Are these good reasons?Plausibility (good. lower for commits)

• Best properties? (depend on the project)

15

example: CmSm - Freecol

Are these good reasons?Plausibility (good. lower for commits)

• Best properties? (depend on the project)

15

example: CmSm - Freecol

Are these good reasons?Plausibility (good. lower for commits)

• Best properties? (depend on the project)

15

example: CmSm - Freecol

Conclusion• Finding automatically the reason for co-changes IS

POSSIBLE!

• Clusters provide better plausibility

• Commits provide better coverage

• Both properties provide better reasons for commits, unclear for clusters

16

Explaining why methods change together

More info: Angela Lozano alozano@soft.vub.ac.be

17