Upload
yida-tao
View
255
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Presented at FSE2012 http://www.sigsoft.org/fse20/
Citation preview
How Do Software Engineers Understand Code Changes? An Exploratory Study in Industry
Yida Tao (HKUST), Yingong Dang (MSRA), Tao Xie (NCSU)
Dongmei Zhang (MSRA), Sunghun Kim (HKUST)
2
3
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
4
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
5
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
6
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char by char, javascript has awesome string
methods… ”
7
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char by char, javascript has awesome string
methods… ”
8
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char by char, javascript has awesome string
methods… ”
“…”
“…”
“…”
9
“How do software engineers understand code changes?”
10
Research Questions RQ1: How frequent is code change
understanding practiced and in which development tasks it is required?
RQ2: What are engineers’ information needs and difficulty for understanding code changes?
RQ3: How to improve the effectiveness and efficiency of the practices in understanding code changes?
11
Study Methodology
Literature Review
• Potential information needs
Questionnaire Design
• Investigate RQ1, RQ2
Pilot Interview
• Question is relevant & clear
12
Study Methodology
Literature Review
• Potential information needs
Questionnaire Design
• Investigate RQ1, RQ2
Pilot Interview
• Question is relevant & clear
Online Survey
• 16% response rate (180 respondents)
Follow-up Interview
• Investigate RQ3
Analysis
• Answering RQs
13
Survey Participants
Dev55%
Test31%
PM14%
Role Distribution Product Team
OS
Desktop App
Web App
Mobile App
Service
Others
14
RQ1 • Frequency ?• Development tasks ?
RQ2 • Information needs ?• Difficulty ?
RQ3 • Improvement ?
RQs
15
RQ1: Frequency of Understanding Code Changes
How often do you need to understand code changes?o Several times each houro About once an houro Several times each dayo About once a dayo Several times each weeko About once a weeko Rarelyo Never
16
RQ1: Frequency of Understanding Code Changes
Several times each hour
About once an hour
Several times each day
About once a day
Several times each week
About once a week
RarelyNever
05
101520253035404550
DevTestPM
Abso
lute
# o
f re
sponse
s
17
RQ1: Tasks Requiring Code Change Understanding
“Select the top three tasks that most often require you to understand code changes” [Design/Planning] Refactoring [Implementation] Developing new feature [Implementation] Fixing bug [Integration] Resolving merge conflict [Verification] Reviewing others code changes [Verification] Reviewing my own code changes [Verification] Writing & updating test cases Other, please specify
18
RQ1: Tasks Requiring Code Change Understanding
Reviewing others' changes
Fixing bug
Developing new feature
Reviewing my own changes
Writing/updating test cases
Refactoring
Resolving merge conflict
0% 15% 30% 45% 60% 75%
121
100
89
73
48
34
30
Percentage of participants who select the task
19
RQ1• Frequently practiced• Major development
tasks
RQ2 • Information needs ?• Difficulty ?
RQ3 • Improvement ?
Answers to RQs
20
Potential Information Needs Literature review (code-change analysis and
management) 180 articles in 10 SE venues over the past decade
21
Potential Information Needs Literature review (code-change analysis and
management) 180 articles in 10 SE venues over the past decadeReasoning & assessing the
change• Completeness• Clones• Design• …Exploring the change’s context & impact• Risk• Consistency• Tests• ...
Evaluating the change history• Change-proneness• Defect-proneness
22
Survey Questions “Rate the importance & difficulty of each
information need (formulated as question) in a change understanding task”
Very Importa
nt
Important
Somewhat
ImportantNot
Important
3
2
1
0
23
Survey Questions “Rate the importance & difficulty of each
information need (formulated as question) in a change understanding task”
Very Importa
nt
Important
Somewhat
ImportantNot
Important
Very Difficult
Difficult
Relatively Easy
Straightfor-ward
3
2
1
0
24
Survey Questions “Rate the importance & difficulty of each
information need (formulated as question) in a change understanding task”
Does this change introduce code clones?
Does this change break any code elsewhere?
Which tests should be run to verify this change?
Is this changed location a hotspot for past fixes?……
Very Importa
nt
Important
Somewhat
ImportantNot
Important
Very Difficult
Difficult
Relatively Easy
Straightfor-ward
3
2
1
0
25
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
26
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
RiskConsiste
ncy
Completeness
Design
27
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
RiskConsiste
ncy
Completeness
Design
Rationale
28
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
RiskConsiste
ncy
Completeness
Design
Rationale
Change-proneness
Defect-proneness
29
RQ1• Frequently practiced• Major development
tasks
RQ2• Risk & Quality are
important but difficult to know
RQ3 • Improvement ?
Answers to RQs
30
RQ3: Interview Items
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
Risk
Rationale
Change-proneness
Defect-proneness
31
Determining a Change’s Risk
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
Risk
Current Practice on Determining a Change’s Risk
32
Manual Code Review• Error-prone• Cross-components• Unclear interface• Hidden assumptions• …
Unit & Regression Testing• Time consuming• Depends on how thorough the tests are• …
33
Support Determining a Change’s RiskM
anual co
de
revie
w • Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change
34
Support Determining a Change’s RiskM
anual co
de
revie
w • Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change
Diff
Code Analy
sis
Navigation in diff
“…miss a level of understanding object
relationships”
35
Support Determining a Change’s RiskTe
stin
g • which code must be retested as it is dependent upon the change?
• who owns testing that dependency?
• which tests must be run?
“An ‘Intelli-sense’ for updating these (affected) tests would be nice as well.”
36
Discussion
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
Rationale
Change-proneness
Defect-proneness
Discussion
37
• Availability & Quality of commit message• “It’s entirely up to the dev making the change as to
how hard or easy it is for someone else to figure out why the change was made.”
Why is understanding the rationale of a change easy?
• Developers• Here and now• Short-term issue• Own Knowledge
• Testers & PMs• Historical metrics might be good to reflect
bugginess and complexity of a specific area
Why are historical metrics not that important?
38
Other Information Needs
“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”
39
Other Information Needs
“Can this change be broken into smaller discreet
changes?”
“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”
40
Composite Code Change
41
Understanding a Composite Code Change
42
Decomposing a Composite Code Change
43
RQ1• Frequently practiced• Major development
tasks
RQ2• Risk & Quality are
important but difficult to know
RQ3• Determining a
change’s risk• Decomposing a
composite change
Answers to RQs
44
Summary Evidence
Understanding code changes is a fundamental practice that happens frequently in major development tasks
45
Summary Evidence
Understanding code changes is a fundamental practice that happens frequently in major development tasks
Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change
46
Summary Evidence
Understanding code changes is a fundamental practice that happens frequently in major development tasks
Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change
Practical Needs Navigation in diff Change decomposition Available & informative commit message
47
Acknowledgment
All participants of survey / interview
Miryung Kim, Robin Moeur, Thomas Zimmermann, Jacek Czerwonka, and Kathryn McKinley