Object-Oriented Reengineering Patterns — an Overview Oscar Nierstrasz Software Composition Group scg

Object-Oriented Reengineering Patterns — an Overview

Oscar NierstraszSoftware Composition Groupwww.iam.unibe.ch/~scg/

http://www.iam.unibe.ch/~scg/

What is wrong with this picture?What is wrong with this picture?

© Oscar Nierstrasz

JUGS — Object-Oriented Reengineering Patterns

3

Roadmap

> Object-Oriented Legacy Systems> Reengineering patterns> Reengineering tools and techniques> Focus on Change

© Oscar Nierstrasz


4

Roadmap

> Object-Oriented Legacy Systems— Software Evolution — The cost of change— The Reengineering lifecycle

> Reengineering patterns> Reengineering tools and techniques> Focus on Change

© Oscar Nierstrasz


5

Lehman’s Laws

Continuing change— A program that is used in a real-world environment must

change, or become progressively less useful in that environment.

Increasing complexity— As a program evolves, it becomes more complex, and extra

resources are needed to preserve and simplify its structure.— Lehman and Belady, 1985

© Oscar Nierstrasz


6

The Dilemma of Legacy Software

A legacy system is a piece of software that:— you have inherited, and— is valuable to you.

Symptoms— Loss of knowledge— Architecture & design drift— Hard to make changes— …

You can’t afford to throw it out, but it is too expensive to change

You can’t afford to throw it out, but it is too expensive to change

© Oscar Nierstrasz


7

OO Legacy

> Object-oriented legacy systems are successful OO systems whose architecture and design no longer respond to changing requirements

© Oscar Nierstrasz


8

cost

time

cost

time

The cost of change

x 200

We need to reduce the cost of change over time …We need to reduce the cost of change over time …

— cf., XP Explained

© Oscar Nierstrasz


9

FAMOOS Case Studies

Domain LOC Reengineering Goalpipeline planning 55,000 extract design

user interface 60,000 increase flexibility

embedded switching 180,000 improve modularity

mail sorting 350,000 portability & scalability

network management 2,000,000 unbundle application

space mission 2,500,000 identify components

Different reengineering goals … but common themes and problems !Different reengineering goals … but common themes and problems !

© Oscar Nierstrasz


10

The Reengineering Life-Cycle

Designs

Code

Requirements

Need appropriate methods and toolsNeed appropriate methods and tools

© Oscar Nierstrasz


11

Roadmap

> Object-Oriented Legacy Systems> Reengineering patterns

— Design patterns vs. process patterns — A Map of Reengineering Patterns— A whirlwind tour!

> Reengineering tools and techniques> Focus on Change

© Oscar Nierstrasz


12

Reengineering Patterns

In software engineering, a design pattern is a general solution to a common problem in software design. A design pattern isn't a finished design that can be transformed directly into code; it is a description or template for how to solve a problem that can be used in many different situations.

In software engineering, a design pattern is a general solution to a common problem in software design. A design pattern isn't a finished design that can be transformed directly into code; it is a description or template for how to solve a problem that can be used in many different situations.

Reverse engineering patterns encode expertise and trade-offs in extracting design from source code, running systems and people.

—Even if design documents exist, they are typically out of sync with reality.

Reengineering patterns encode expertise and trade-offs in transforming legacy code to resolve problems that have emerged.

—These problems are typically not apparent in original design but are due to architectural drift as requirements evolve

© Oscar Nierstrasz


13

A Map of Reengineering Patterns

Tests: Your Life Insurance

Detailed Model Capture

Initial Understanding

First Contact

Setting Direction

Migration Strategies

Detecting Duplicated Code

Redistribute Responsibilities

Transform Conditionals to Polymorphism

© Oscar Nierstrasz


14

Setting Direction

Agree on Maxims

Appoint aNavigator

Speak to theRound Table

Fix Problems,Not Symptoms

If It Ain't BrokeDon't Fix It

Keep it Simple

Set direction

Maintaindirection

Coordinatedirection

Where to start

What not to doWhat to do

How to do itPrinciples & Guidelines for Software project management especially relevant for reengineering projects

Principles & Guidelines for Software project management especially relevant for reengineering projects

Most Valuable First

© Oscar Nierstrasz


15

Most Valuable First

Problem: Which problems should you focus on first?

Solution: Work on aspects that are most valuable to your customer

> Maximize commitment, early results; build confidence> Difficulties and hints:

— Which stakeholder do you listen to?— What measurable goal to aim for?— Consult change logs for high activity— Play the Planning Game— Wrap, refactor or rewrite? — Fix Problems, not Symptoms

Vs. Fix the Buggiest First?

Vs. Fix the Buggiest First?

© Oscar Nierstrasz


16

First Contact

System experts

Chat with theMaintainers

Interviewduring Demo

Talk withdevelopers

Talk withend users

Software System

Read All the Codein One Hour

Do a MockInstallation

Read it Compile it

Skim theDocumentation

Talk about it

Verify whatyou hear Read

about it

Feasibility assessment (one week time)Feasibility assessment (one week time)

© Oscar Nierstrasz


17

Initial Understanding

Top down

Speculate about Design

Recover design

Analyze the Persistent Data

Study the Exceptional Entities

Read it Compile it

Bottom up

understand Obtain a higher-level modelunderstand Obtain a higher-level model

© Oscar Nierstrasz


18

Pattern: Study the Exceptional Entities

Problem— How can you quickly gain insight into complex software?

Solution— Measure software entities and study the anomalous ones

Steps— Use simple metrics— Visualize metrics to get an overview— Browse the code to get insight into the anomalies

© Oscar Nierstrasz


19

System Complexity View

Nodes = ClassesEdges = Inheritance Relationships

Width = Number of AttributesHeight = Number of MethodsColor = Number of Lines of Code

System Complexity View

ColorMetric

PositionMetrics

Width Metric

Height Metric

© Oscar Nierstrasz


20

Detailed Model Capture

Tie Code and Questions

Refactor to UnderstandKeep track of

your understanding

Expose design

Step through the Execution

Look for the Contracts

Learn from the Past

Expose collaborations

Expose contracts

Expose evolution

Write Teststo Understand


Expose the design & make sure it remains exposedExpose the design & make sure it remains exposed

• Use Your Tools• Look for Key Methods• Look for Constructor Calls• Look for Template/Hook Methods• Look for Super Calls

• Use Your Tools• Look for Key Methods• Look for Constructor Calls• Look for Template/Hook Methods• Look for Super Calls

© Oscar Nierstrasz


21

Tests: Your Life Insurance

Write Tests to Enable Evolution

Grow Your TestBase Incrementally

Managing testsUse a Testing

Framework

Test the Interface,Not the Implementation

Record BusinessRules as Tests

Designingtests


• Test Fuzzy features• Test Old Bugs• Retest Persistent Problems

• Test Fuzzy features• Test Old Bugs• Retest Persistent Problems

Regression Testafter Every Change


Migration StrategiesMigration Strategies

© Oscar Nierstrasz


22

Migration

Migrate SystemsIncrementally

ConserveFamiliarity

HowUse Profiler

before Optimizing

Build ConfidenceInvolve the UsersHow Why

Why

Prototype theTarget Solution

Always Have aRunning Version


Present theRight Interface

Distinguish Publicfrom Published Interfaces

DeprecateObsolete Interfaces

Make a Bridgeto the New Town

How

Tests, yourLife-Insurance

Tests, yourLife-Insurance

Where to

© Oscar Nierstrasz


23

Detecting Duplicated Code

Compare Code Mechanically

Visualize Code as Dotplots


Redistribute Responsibilities Transform

Conditionals to Polymorphism


Detect

Understand

© Oscar Nierstrasz


24

Pattern: Visualize Code as Dotplots

Problem— How can you effectively identify significant duplication in a

complex software system?

Solution— Visualize the code as a dotplot, where dots represent

duplication.

Steps— Normalize the source files— Compare files line-by-line— Visualize and interpret the dotplots

© Oscar Nierstrasz


25

File A

File A

File B

File B

Clone detection by string-matching

Solid diagonals indicate significant duplication between or within source files.

Solid diagonals indicate significant duplication between or within source files.

© Oscar Nierstrasz


26

Dotplot Visualization

Exact Copies Copies with Inserts/Deletes Repetitive

a b c d e f a b c d e f a b c d e fa b x y e f b c d e a b x y dc ea x b c x d e x f xg ha

Variations Code Elements(Helfman, 1995)

Sample Dot Configurations:

© Oscar Nierstrasz


27


Eliminate Navigation Code

Data containers

Monster client ofdata containers

Split Up God Class

Move Behaviour Close to Data

Chains of datacontainers

© Oscar Nierstrasz


28

High-level refactorings

QuickTime™ and aNone decompressor

are needed to see this picture.

High-level refactorings make use of many low-

level refactorings

High-level refactorings make use of many low-

level refactorings

© Oscar Nierstrasz


29

TransformSelf Type Checks

Test providertype

Test self typeTest external

attribute

TransformClient Type Checks

Transform Conditionalsinto Registration

Testnull values Introduce

Null Object

Factor Out Strategy

Factor Out State

Test object state


© Oscar Nierstrasz


30

Roadmap

> Object-Oriented Legacy Systems> Reengineering patterns> Reengineering tools and techniques

— MOOSE — A reengineering platform— History as an entity

> Focus on Change

© Oscar Nierstrasz


31

The Components of Moose

Smalltalk (VW)

Browsing

Metrics

Querying

Grouping

Extensible meta model

Model repository

Smalltalk

Java

C++

…

CVS

SVN

MSEExternalParser

Mondrian

Dyna Van ...Hapax

CodeCrawler

© Oscar Nierstrasz


32

Moose Browser

© Oscar Nierstrasz


33

Moose visualizations

© Oscar Nierstrasz


34

Moose offers metrics and navigation

© Oscar Nierstrasz


35

Class Blueprint — Example

> Root Class:—Template—Inconsistent accessor definition—Unused accessors—Direct attribute access—Template Method (DP)

> Subclasses:—Siamese twins—Pure overriders

© Oscar Nierstrasz


36

Moose offers many visualizations

© Oscar Nierstrasz


41

Moose is an environment

> Refactoring engine> Static and dynamic

architecture recovery> Trace-based feature recovery> Concept mining> Duplication detection> Evolution analysis> …

With a cast of thousands …Tobias AebiGabriela ArévaloMihai BalintFrank BuchliThomas BühlerPhilipp BungeCalogero ButeraMarco D’AmbrosSerge DemeyerStéphane DucasseFredi FrankMichael FreidigTudor GîrbaGeorges GolomingiOrla GreevyDavid GurtnerMatthias JunkerMarkus HofstetterMarc-Philippe Horvath

Markus Kobe Adrian KuhnMichele LanzaAdrian LienhardMircea LunguPietro MalorgioMichael MeyerLaura PonisioDaniel RatiuTamar RichnerMatthias RiegerDaniel SchweizerMauricio SeebergerLukas SteigerMarc StettlerDaniele TalericoSander TichelaarChistoph WysseierRicky Wettel

© Oscar Nierstrasz


42

Roadmap

> Object-Oriented Legacy Systems> Reengineering patterns> Reengineering tools and techniques> Focus on Change

— Concluding remarks

© Oscar Nierstrasz


43

Object-Oriented Reengineering

> Yes, there are object-oriented legacy systems too!

— … which is a sign of health!

> Reverse engineering and reengineering are essential activities in the lifecycle of any successful software system.

— … consequently, do not consider it second class work

> There is a large set of lightweight tools and techniques to help you with reengineering.

— … the list is growing

> Nevertheless, people must do the job!— … so pick them carefully and reward them

appropriately!

© Oscar Nierstrasz


44

Trends

> Software is evolving ever more rapidly— Faster, smaller hardware

– leads to new application domains

— Globalization– leads to new opportunities and requirements

— The internet and mobile computing– raise the bar for interoperability

We cannot afford to continue to develop software in the same old way!

We cannot afford to continue to develop software in the same old way!

© Oscar Nierstrasz


45

Conclusion

We need to place software evolution at the centre of our software processes and tools.

> Methods— Empirical research into “best practices” to support change

– Tie forward, reverse and re-engineering

> Tools— Model, analyse and transform evolving systems

– Co-evolution of requirements, design and code

> Programming Languages— Software as living systems

– Context-aware programs

Questions?Comments?Questions?Comments?

Documents

Object-Oriented Reengineering Patterns — an Overview Oscar Nierstrasz Software Composition Group scg