Clone Digger - discovers duplicate code in Python.

Sangharsh Agarwal

What is Clone Digger?

• Clone Digger aimed to detect similar code in Python and Java programs. The synonyms for the term "similar code" are "clone" and "duplicate code".

• What is software clone?Two continuous fragments of code form clone if they are similar enough.

• Why is it important to detect clones?The presence of clones can increase maintenance cost of the code. Detected clones can be refactored or just kept in mind.

Why is it important to detect code clones?• 5% - 20% of code in software systems are

clones.

• Why do programmers produce clones?– Development strategy

– Maintenance benefits

– Overcoming underlying limitations

– Cloning by accident

• Why is the presence of code clones bad?– Errors in the original must be fixed in every clone.

Why to use Clone Digger?

• Why should I use Clone Digger to detect clones?There are several clone detection tools, they are listed here. The benefits of Clone Digger are:1. Variety of handled clone types. Strictly speaking, a

pair of sequences of statements is considered a clone if one sequence can be obtained from the other by replacing some small sub expressions. Particularly, changes of variable and function names and constants are allowed.

2. It's free (provided under the GPL license).

Definition of clone

• Different clone definitions can be classified according to the level of granularity:– List of strings– Sequence of tokens– Abstract syntax trees (AST)– Semantic information

• Clone Digger work on the AST level• Clone Digger consider two sequences of statements

as a clone if one of them can be obtained from the other by replacing some sub trees.

Quick Start

• $ easy_install clonedigger

• $ clonedigger –o output.html <source_path>

• $ firefox output.html

• Additional parameters such as thresholds can be also set (use --help to know more)

Clone Digger – Run from command prompt -1 (pytests/xdcr)[root@cen-1413 testrunner]# sudo /root/.local/bin/clonedigger -o output.html pytests/xdcr/

Parsing pytests/xdcr/upgradeXDCR.py ... done

Parsing pytests/xdcr/xdcrbasetests.py ... done

Parsing pytests/xdcr/biXDCR.py ... done

Parsing pytests/xdcr/esXDCR.py ... done

Parsing pytests/xdcr/esbasetests.py ... done

Parsing pytests/xdcr/uniXDCR.py ... done

Parsing pytests/xdcr/checkpointXDCR.py ... done

Parsing pytests/xdcr/observexdcrtest.py ... done

Parsing pytests/xdcr/__init__.py ... done

Parsing pytests/xdcr/xdcrMiscTests.py ... done

Parsing pytests/xdcr/rebalanceXDCR.py ... done

Parsing pytests/xdcr/pauseResumeXDCR.py ... done

376 sequences

average sequence length: 5.752660

maximum sequence length: 69

Number of statements: 2163

Calculating size for each statement... done

Building statement hash... done

Number of different hash values: 284

Building patterns... 1000, 2000, 601 patterns were discovered

Choosing pattern for each statement... 1000, 2000, done

Finding similar sequences of statements...

-----------------------------------------

Warning: sequence of statements starting at pytests/xdcr/xdcrbasetests.py:33

consists of many similar statements.

It will be ignored. Use --force to override this restriction.

Please refer to http://clonedigger.sourceforge.net/documentation.html

-----------------------------------------

369 sequences were found

Refining candidates... 167 clones were found

Removing dominated clones... -36 clones were removed

Plugins for Clone Digger

• Integration with Hudson

See a short manual by Pavol Zibrita to learn how to get Clone Digger and Hudson (continuous integration tool) to work together.

• Eclipse Plugin

Eclipse Plugin for Clone Digger has been implemented during Google Summer of Code 2008. See the manual to learn how install it and how to work with it.

What to do with found clones?

• Remove clones by refactoring. Extract method and Pull Up method can be used.

• Detect library candidates.

• Search for bugs.

References

• http://clonedigger.sourceforge.net/index.html

• http://clonedigger.sourceforge.net/download.html

• http://wiki.hudson-ci.org/display/HUDSON/Meet+Hudson