Embed Size (px)
DESCRIPTION
When developers perform a software maintenance task, they need to identify artifacts—e.g., classes or more specifically methods—that need to be modified. To this aim, they can browse various kind of artifacts, for example use case descriptions, UML diagrams, or source code. This paper reports the results of a study—conducted with 33 participants— aimed at investigating (i) to what extent developers use different kinds of documentation when identifying artifacts to be changed, and (ii) whether they follow specific navigation patterns among different kinds of artifacts. Results indicate that, although developers spent a conspicuous proportion of the available time by focusing on source code, they browse back and forth between source code and either static (class) or dynamic (sequence) diagrams. Less frequently, developers—especially more experienced ones—follow an “integrated” approach by using different kinds of artifacts.
Citation preview
An Empirical Investigation on Documentation Usage Patterns in
Maintenance Tasks
Gabriele Gerardo Massimiliano Rocco Sebastiano Bavota Canfora Di Penta Oliveto Panichella
Context and Motivations • Software Development
Case Study • User Study with 33 Partecipants
Results •Documentation usage pattern during maintenance evolution
actvity
Software Development
Maintenance tasks are generally facilitated when software
documentation is available
Past Work
The use of UML diagram significantly improves the
functional correctness of changes
E. Arisholm et al. TSE 2006
Past Work
Help to reduce the time needed to understand ho
perform maintenance tasks by approximately 20 %
Eirik Tryggeseth EMSE 1997
How such documentation is browsed by developers to perform
maintenance activities?
Empirical Study
Goal: observe the software documents browsed by Developers during maintenance tasks Quality focus: understand the documentation usage pattern during maintenance activities Perspective: researchers interested to develop tool to improve the program comprehension
• RQ1: How much time did participants spend on different kinds of artifacts?
• RQ2: How do participants navigate
different kinds of artifacts to
identify code to be changed
during the evolution task?
Research Questions
Context • Object: software artifacts from SMOS, a school
automation system developed by graduate students at the University of Salerno (Italy).
• Subjects: 33 partecipants
11 Bachelor Students 18 Master Students 4 PhD Students
Study Procedure
Maintenance tasks
Bug Fixing:
Add a new feature:
Improve existing features:
Example: Maintenance task
Monitoring Maintenance Activity
Monitoring Maintenance Activity
TIMESTAMP
Data Collection XML log
Files
Data Collection XML log
Files
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1 R
eca
ll
Pre
cisi
on
F-M
eas
ure
Re
call
Pre
cisi
on
F-M
eas
ure
Re
call
Pre
cisi
on
F-M
eas
ure
Undergraduates Graduates ALL
Subjects Performances
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1 R
eca
ll
Pre
cisi
on
F-M
eas
ure
Re
call
Pre
cisi
on
F-M
eas
ure
Re
call
Pre
cisi
on
F-M
eas
ure
Undergraduates Graduates ALL
Subjects Performances
Average 0.75 FM
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1 R
eca
ll
Pre
cisi
on
F-M
eas
ure
Re
call
Pre
cisi
on
F-M
eas
ure
Re
call
Pre
cisi
on
F-M
eas
ure
Undergraduates Graduates ALL
Subjects Performances
Average 0.75 FM
+ 5% FM for Graduate
Students
RQ1
How much time did participants
spend on different kinds of artifacts?
Javadoc
Use Case
Class Diagram
Sequence Diagram
Source Code
ALL
Graduates
Undergraduates
15%
Use of each artifacts in overall tasks
60% 33%
100%
72%
RQ1: How much time did participants spend on different kinds of artifacts?
72%
13% 10% 3% 2%
72%
13% 10% 3% 2%
RQ1: How much time did participants spend on different kinds of artifacts?
Undergraduates students used Source Code and Javadoc significantly
more than Graduate students
Undergraduate Students
72%
13% 10% 3% 2%
RQ1: How much time did participants spend on different kinds of artifacts?
Graduate Students
Graduate students used Class Diagrams significantly more than Undergraduates
Perceived Importance of Software Artifacts
1
3
4
4
2
3
4
5
4
2
0 2 4 6
Use Case
Class Diagram
Sequence Diagram
Source Code
Javadoc
Graduates
Undergraduates
Perceived Importance of Software Artifacts
1
3
4
4
2
3
4
5
4
2
0 1 2 3 4 5 6
Use Case
Class Diagram
Sequence Diagram
Source Code
Javadoc
Graduates
Undergraduates
Sequence diagram represents a fair compromise between use cases (too abstracts) and class diagrams (providing useless details about an entire subsystem)..
Perceived Importance of Software Artifacts
1
3
4
4
2
3
4
5
4
2
0 1 2 3 4 5 6
Use Case
Class Diagram
Sequence Diagram
Source Code
Javadoc
Graduates
Undergraduates
Sequence diagram represents a fair compromise between use cases (too abstracts) and class diagrams (providing useless details about an entire subsystem)..
With the other sources of documentation available Javadoc became useless to identify impacted components
Summary RQ1
Summary RQ1 1) Participants spent more time to analyze low-level artifacts
as compared to high-level artifacts 2) Participants consider the most important sources of
information sequence and class diagrams 3) Undergraduate students spent a significantly higher
proportion of time on source code than graduate students that, instead, spent more time on class diagrams
Summary RQ1 1) Participants spent more time to analyze low-level artifacts
as compared to high-level artifacts 2) Participants consider the most important sources of
information sequence and class diagrams 3) Undergraduate students spent a significantly higher
proportion of time on source code than graduate students that, instead, spent more time on class diagrams
Summary RQ1 1) Participants spent more time to analyze low-level artifacts
as compared to high-level artifacts 2) Participants consider the most important sources of
information sequence and class diagrams 3) Undergraduate students spent a significantly higher
proportion of time on source code than graduate students that, instead, spent more time on class diagrams
RQ2
How do participants navigate different kinds
of artifacts to identify code to be changed
during the evolution task?
3%
Starting Point: Graduate vs Undergraduate students
12% 17%
25%
42%
Starting Point: Graduate vs Undergraduate
Undergraduate
Graduate
58%
32% 24%
26%
9%
22%
3%
17%
4% 3%
18%
8%
2%
2%
1%
1%
3%
0%
1%
0%
4%
12%
16%
12%
10%
4%
4%
1%
2%
1%
1%
5%
S
D
(SD)+
(US)+
U(SD)+
(DS)+
J
U
S(US)+
SU(SD)+
Other
Graduate
Undegraduate
S= Sequence Diagram
D= Class Diagram
U= Use Case
J= Javadoc
Most frequents patterns before reaching source code
18%
8%
2%
2%
1%
1%
3%
0%
1%
0%
4%
12%
16%
12%
10%
4%
4%
1%
2%
1%
1%
5%
0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20%
S
D
(SD)+
(US)+
U(SD)+
(DS)+
J
U
S(US)+
SU(SD)+
Other
Graduate
Undegraduate
S= Sequence Diagram
D= Class Diagram
U= Use Case
J= Javadoc
Most frequents patterns before reaching source code
18%
8%
2%
2%
1%
1%
3%
0%
1%
0%
4%
12%
16%
12%
10%
4%
4%
1%
2%
1%
1%
5%
0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20%
S
D
(SD)+
(US)+
U(SD)+
(DS)+
J
U
S(US)+
SU(SD)+
Other
Graduate
Undegraduate
More experienced participants use a more “Integrated approach”
S= Sequence Diagram
D= Class Diagram
U= Use Case
J= Javadoc
Most frequents patterns before reaching source code
Source Code
Sequence Diagram
Javadoc
Transition Graph between kinds of Software Artifacts
Source Code
Sequence Diagram
Javadoc
1) From Source Code participants in most cases “go back” to
Sequence and Class Diagrams
2) From Sequence and Class Diagrams
participants in most cases “go back” to
Source Code
3) Starting from a Use Case, participants go
ahead reading Sequence Diagrams. Only after,
they reading and writing Source Code
Transition Graph between kinds of Software Artifacts
0%
20%
11%
30%
16%
61%
41%
70%
12%
15%
60%
24%
58%
54%
0% 10% 20% 30% 40% 50% 60% 70% 80%
USDC
USD
USC
UDC
SDC
SC
DC
Graduates
Undergraduates
1) Developers never used Javadoc
Most Frequents Patterns
2) Patterns end always with Source Code
S= Sequence Diagram
D= Class Diagram
U= Use Case
J= Javadoc
C= Source Code
Summary RQ2 • 1) Participants tend to start the assigned task from source
code or from design documents , i.e., class and sequence diagrams
• 2) More experienced participants tend to follow a more
integrated approach than less experienced Ones • 3) During tasks, participants tend to go back and forth
repeatedly between source code and to design diagrams (sequence and class diagrams).
…Conclusions…
…Conclusions…
…Conclusions…
…Conclusions…
…Conclusions…
…Future Directions…
