Documenting design pattern instances:A Family of experiments on

Source code comprehensibility

Gabriella Dodero (uniBZ)Joint work with G.Scanniello (uniBAS), C.Gravino, M.Risi,

G.Tortora (uniSA)

RDSS, December 2014

Summary:

1. A research question2. A family of experiments3. Our results

A research question● Consider a software maintenance task, typically done by

someone who is NOT an author of such code.● The system under maintenance contains several

patterns● Is there a difference in such a task, if code is

DOCUMENTED showing use of pattern instances?● That is, not documenting patterns w.r.t. documenting

wherever patterns have been used● Is there a difference between ways of documenting

patterns?● That is, textual w.r.t. graphical documentation

A family of experiments-1 ● Make an experiment about software maintenance, i.e.

give a set of „software comprehension tasks“● Repeat the experiment with different populations

(developers with different experience)● What is measured:

– Correctness of understanding– Time– Confidence– Perceived difficulty

A family of experiments - 2

Choose a piece of software

● A chunk of JHotDraw v5.1 (1326 LOC, non trivial!)

Choose a notation for documenting patterns

● Textual descriptions reported in the code as comments

● Graphic documentation like class diagrams

A family of experiments - 3● „Family“ means replicating an original experiment

under controlled conditions● In this case: using populations with different

experience as developers

Participants Where How many

Practitioners Companies 25

2nd year Msc U.Salerno 25

3rd year Bsc U.Basilicata 23

2-3 yr PhD unibz 15

A family of experiments - 4● In each experiment,

● 1/3 of participants were given textual documentation, ● 1/3 were given graphical documentation, and● 1/3 had no documentation on pattern instances

● Hypotheses:(1) There is no difference in Correctness nor Time

between people having documentation and not having documentation

(2)There is no difference between people with different experience

A family of experiments - 5● What was asked: 14 questions like this one:

● A post assessment questionnaire on the experience● Typical duration was 2-3 hours

Results - 1● Hypothesis (1) is rejected for Correctness of

understanding, by professionals and PhD students only

– Time is not affected by documentation● But there is no significant difference between

those who had textual documentation or graphical documentation

● Hypothesis (2) is rejected for Experience, with respect to Correctness of Understanding and Time

Results - 2● Correctness of understanding is significantly greater when

pattern instances are correctly identified● 30% improvement on Correctness of understanding

● Correctness of understanding is greater also for Msc students, when patterns are documented, but this is not statistically significant

● Experienced participants (professionals, PhD) benefit more than non-experienced participants (BSc, Msc students) from the documentation on patterns

● Experienced participants with textually documented patterns were the fastest to do the task

Results - 3● Lessons for practitioners:

● Documenting instances improves code comprehensibility, for maintainers with experience

● Reading such documentation does not slow down comprehension tasks

● Comprehensibility improves when pattern instances are correctly identified, when developers have at least a Bsc degree

● Lessons for researchers:● Bsc or Msc students may be too inexperienced

compared to professionals, to gather reliable results by performing experiments on them!!!

Thank You!Questions?

Gdodero @ unibz.it