Software Modularity: Paradoxes, Principles, and Architectures

Andrzej Olszak

Jaroslav Tulach

Andrzej Olszak

• Ph.D. Research fellow @ SDU

• Creator of Featureous tool

• Fights cancer @ Dako

Jaroslav Tulach

• Founder and architect of NetBeans IDE and RCP

• API designer

• Speaker and author

http://paradoxes.apidesign.org

Agenda

1. Dividing into modules (Andrzej):

1. Architectures

2. Principles

2. Designing module APIs (Jaroslav):

1. Paradoxes

2. Principles

Motivation

Getting out of the monolithic cave

The starting point

A monolithic system

• No apparent logical parts– Everything changes

together

– Difficult to control complexity

The goal

A modular system

• Decomposed into logical modules

• Modules partial:– Comprehension

– Change

The tool

A module system

• Wraps logical modules into physical modules– a.k.a. advanced JARs

– Enforce code isolation

A possible result

• Module system ⇏Modular code– It all depends on how you design

the modules

Part I

Architectures

A running example

• Let’s assume that this is our system

– 3 layers of code that provide 3 features to the users

• This is the essence of your architecture if you use:

– MVC, MVP, Onion arch, Hexagonal arch, …

“UI”

“Logic”

“Domain Model”

Modularizing layers

• There are at lease three options:

View

Controller

Model

Modularizing layers

• There are at lease three options:

View

Controller

Model

View

Controller

Model

App

feat

ure

1

feat

ure

2

feat

ure

3

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

2. Migrate UI to JavaFX1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

2. Migrate UI to JavaFX1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

2. Migrate UI to JavaFX1. Modify user functionality

Choosing ‘best’ modularization

• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]

• Let’s consider two change scenarios:

View

Controller

Modelfe

atu

re1

feat

ure

2

feat

ure

3

2. Migrate UI to JavaFX1. Modify user functionality

No silver-bullet modularization

Anticipating future changes

Feature-oriented evolutionary changes

75%

Other evolutionary changes25%

Anticipating future changes

Software costs in organizations [Moad’90, Nose’90, Erli’00]

Evolution &maintenance

60-90%

Initial development10-40%

Feature-oriented evolutionary changes

75%

Other evolutionary changes25%

Anticipating future changes

Software costs in organizations [Moad’90, Nose’90, Erli’00]

Modularization in practice:the case of NDVis

• Neurological analysis tool by VisiTrend

– Monolithic -> NetBeans MS

– Improve functional customizability

– Improve reusability of core algorithms

• Starting point– 10 KLOC, 27 use cases

– Unfamiliar source code

– Complex and unfamiliar domain

View

Controller

Model

feat

ure

1

feat

ure

2

feat

ure

3

Feature-oriented restructuring

Modularizing features with Featureous tool

Modularizing features with Featureous tool

1. Feature location(a.k.a. traceability)

Modularizing features with Featureous tool

1. Feature location(a.k.a. traceability)

2. Feature-orientedanalysis

Modularizing features with Featureous tool

1. Feature location(a.k.a. traceability)

2. Feature-orientedanalysis

3. Iterative Restructuring

4. Module APIs

Modularization @ 35 man-hours

• Explicit and pluggable features

• Reusable core

Part II

Principles

Separation of Concerns

• “To study an aspect of a subject matter in isolation”[Dijk’74]

• Software consists of “concerns”– Features

– Persistence

– Security

– Caching

...

• Refined by AspectJ [Kicz’96] and Hyper/J [Tarr’99]

– Multiple dimensions of concern – one dominant

– Scattering & Tangling

Reducing Scattering & Tangling

• Low Scattering

– “a concern implemented by few modules“

– reduces change scope and delocalization [Leto’86]

View

Controller

Model

feature 1

scattering

Scattering & Tangling

• Low Scattering

– “a concern implemented by few modules“

– reduces change scope and delocalization [Leto’86]

• Low Tangling

– "modules dedicated to a single concerns“

– reduces change propagation and interleaving [Ruga’95]

View

Controller

Model

feature 2

feature 1

tangling

scattering

A real-world metaphor

A real-world metaphor

http://xkcd.com/657/

Mapping SoC to other principles

SEPARATION OF CONCERNS

Mapping SoC to other principles

SEPARATION OF CONCERNS

LOW SCATTERING LOW TANGLING

Mapping SoC to other principles

SEPARATION OF CONCERNS

Low coupling[Stev’74]

Information hiding[Parn’72]

DRY[Hunt’99]

LOW SCATTERING LOW TANGLING

Mapping SoC to other principles

SEPARATION OF CONCERNS

Low coupling[Stev’74]

High cohesion[Stev’74]

Information hiding[Parn’72]

DRY[Hunt’99]

Single responsibility[Mart’02]

Common closure[Mart’02]

LOW SCATTERING LOW TANGLING

Measuring SoC

• Concern location + concern-oriented metrics

– Trialed at Motorola [Simm’06]

• Static analysis + Cohesion and Coupling

– Issues: coupling vs. DRY, “uncohesive” java.util

• Repository mining for change-sets:

http://swerl.tudelft.nl/bin/view/Main/TestHistory

clas

ses

Summary

• Module system is a tool, not the goal

• No “silver-bullet” modularization

• Restructuring layers into features is viable

• SoC – the root of all principles

References[Parn’72] Parnas, D.L. (1972). On the criteria to be used in decomposing systems into modules. Communications of the

ACM, 15(12).

[Moad’90] Moad, J. (1990). "Maintaining the competitive edge". Datamation 61-62, 64, 66.

[Erli’00] Erlikh, L. (2000). "Leveraging legacy system dollars for E-business". (IEEE) IT Pro, May/June 2000.

[Nose’90] Nosek, J. & Palvia, P. (1990). "Software maintenance management: changes in the last decade". Journal of Software Maintenance: Research and Practice 2 (3).

[Mart’11] Martin, R.C. (2011). http://blog.8thlight.com/uncle-bob/2011/09/30/Screaming-Architecture.html

[Dijk’74 ] Dijkstra, E. W. (1974). On the role of scientific thought. Selected Writings on Computing: A Personal Perspective.

[Kicz’96] Kiczales, G., Irwin, J., Lamping, J., Loingtier, J., M., Lopes, C., V., Maeda, C. and Mendhekar, A. (1996). Aspect-oriented programming. ACM Computing Surveys, vol. 28.

[Tarr’99] Tarr, P., Ossher, H., Harrison, W. and Sutton, S. M. (1999). N degrees of separation: multi-dimensional separation of concerns. In ICSE’99: Proceedings of the 21st international conference on Software engineering.

[Leto’86] Letovsky, S. and Soloway, E. (1986). Delocalized Plans and Program Comprehension. IEEE Software, 3(3).

[Ruga’95] Rugaber, S., Stirewalt, K. and Wills, L. M. (1995). The interleaving problem in program understanding. In WCRE’95: Proceedings of the 2nd Working Conference on Reverse Engineering.

[Simm’06] Simmons, S., Edwards, D., Wilde, N., Homan, J. and Groble, M. (2006). Industrial tools for the feature location problem: an exploratory study. Journal of Software Maintenance and Evolution Research and Practice, 18(6).

[Mart’02] Martin, R. C. (2002). Agile Software Development, Principles, Patterns, and Practices. Prentice Hall.

[Stev’74] Stevens, W. P., Myers, G. J. and Constantine, L. L. (1974). Structured Design. (E. Yourdon, Ed.)IBM Systems Journal, 13(2).

[Hunt’99] Hunt, A., & Thomas, D. (1999). The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley Professional.

(Many of these papers are available online through Google Scholar, Citeseerx and the authors’ websites)

Feature tracer [backup]

Part III

Paradoxes of API Design

Modularity is relative to change

• API are like stars (paradox 19)

• Designing a universe

• Distributed development (paradox 2)

• Can't know all your users

• Envisioning them via use-cases

• Sustaining (paradox 3)

• One try to get API right

How to anticipate future changes?

• Client vs. Provider APIs (paradox 9)

• Open spaces

• Fixed points

• Stable API evolves

• Backward compatibility (paradox 6)

• Beware of API-less APIs (paradox 8)

• Mediawiki experience

• Loyal customer

Logical vs. Physical Design

• Design oriented on class relationship

• UML, specifications

• Packaging into JARs ignored (paradox 17)

• Influences deployment (paradox 7)

• Defines APIs

• Good common ground (paradox 11)

• Improves your design

A „weight“ of a module

• Environment for a module

• Modules don't live in vacuum

• Expressed by dependencies (paradox 17)

• Weight

• Number & scope of outgoing dependencies

• Less is more (paradox 19)

• Single responsibility principle

Use vs. Re-use

• Kirk Knoernchild

• Monolithic API is easier to use

• Modular API is easier to re-use

• Blackbox pattern (paradox 18)

• OSGi Capabilities

• Good tooling

• Wizards

SOLID APIs

• Single responsibility principle

• Meaning of modifiers (paradox 15)

• Client vs. provider APIs (paradox 9)

• Lightweight API modules

• Open/closed principle

• OK for provider APIs

• Disastrous for client APIs

• Proliferation of instanceof in user code

• Alternative behavior (paradox 16)

SOLID APIs II

• Liskov substitution principle

• AWT Frame extends Component!

• Don't expose deep hierarchies

• Use delegation rather than inheritance

• Client API should be in final classes

• 1:N factory methods

SOLID APIs III

• Interface segregation principle

• Lookup & discover

• OSGi declarative services

• Dependency inversion principle

• Code against interfaces, not implementations

• Does not imply classes are bad (paradox 9)

• Don't fear (injectable) singletons (paradox 14)

API in User Eyes

• Clueless users (paradox 1)

• Have always something else to do

• Evaluation of an API (paradox 4)

• Coolness

• Time to market

• Total cost of ownership

Collaboration

• Maintenance (paradox 10)

• Rely on patches

• Accepting unacceptable (paradox 13)

• Beauty (paradox 5)

• One writer and dozens of users

• Sacrifice the writer

Summary

• http://paradoxes.apidesign.org