Software Modularity: Paradoxes, Principles, and Architectures
Andrzej Olszak
Jaroslav Tulach
Andrzej Olszak
• Ph.D. Research fellow @ SDU
• Creator of Featureous tool
• Fights cancer @ Dako
Jaroslav Tulach
• Founder and architect of NetBeans IDE and RCP
• API designer
• Speaker and author
http://paradoxes.apidesign.org
Agenda
1. Dividing into modules (Andrzej):
1. Architectures
2. Principles
2. Designing module APIs (Jaroslav):
1. Paradoxes
2. Principles
Motivation
Getting out of the monolithic cave
The starting point
A monolithic system
• No apparent logical parts– Everything changes
together
– Difficult to control complexity
The goal
A modular system
• Decomposed into logical modules
• Modules partial:– Comprehension
– Change
The tool
A module system
• Wraps logical modules into physical modules– a.k.a. advanced JARs
– Enforce code isolation
A possible result
• Module system ⇏Modular code– It all depends on how you design
the modules
Part I
Architectures
A running example
• Let’s assume that this is our system
– 3 layers of code that provide 3 features to the users
• This is the essence of your architecture if you use:
– MVC, MVP, Onion arch, Hexagonal arch, …
“UI”
“Logic”
“Domain Model”
Modularizing layers
• There are at lease three options:
View
Controller
Model
Modularizing layers
• There are at lease three options:
View
Controller
Model
View
Controller
Model
App
feat
ure
1
feat
ure
2
feat
ure
3
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
No silver-bullet modularization
Anticipating future changes
Feature-oriented evolutionary changes
75%
Other evolutionary changes25%
Anticipating future changes
Software costs in organizations [Moad’90, Nose’90, Erli’00]
Evolution &maintenance
60-90%
Initial development10-40%
Feature-oriented evolutionary changes
75%
Other evolutionary changes25%
Anticipating future changes
Software costs in organizations [Moad’90, Nose’90, Erli’00]
Modularization in practice:the case of NDVis
• Neurological analysis tool by VisiTrend
– Monolithic -> NetBeans MS
– Improve functional customizability
– Improve reusability of core algorithms
• Starting point– 10 KLOC, 27 use cases
– Unfamiliar source code
– Complex and unfamiliar domain
View
Controller
Model
feat
ure
1
feat
ure
2
feat
ure
3
Feature-oriented restructuring
Modularizing features with Featureous tool
Modularizing features with Featureous tool
1. Feature location(a.k.a. traceability)
Modularizing features with Featureous tool
1. Feature location(a.k.a. traceability)
2. Feature-orientedanalysis
Modularizing features with Featureous tool
1. Feature location(a.k.a. traceability)
2. Feature-orientedanalysis
3. Iterative Restructuring
4. Module APIs
Modularization @ 35 man-hours
• Explicit and pluggable features
• Reusable core
Part II
Principles
Separation of Concerns
• “To study an aspect of a subject matter in isolation”[Dijk’74]
• Software consists of “concerns”– Features
– Persistence
– Security
– Caching
...
• Refined by AspectJ [Kicz’96] and Hyper/J [Tarr’99]
– Multiple dimensions of concern – one dominant
– Scattering & Tangling
Reducing Scattering & Tangling
• Low Scattering
– “a concern implemented by few modules“
– reduces change scope and delocalization [Leto’86]
View
Controller
Model
feature 1
scattering
Scattering & Tangling
• Low Scattering
– “a concern implemented by few modules“
– reduces change scope and delocalization [Leto’86]
• Low Tangling
– "modules dedicated to a single concerns“
– reduces change propagation and interleaving [Ruga’95]
View
Controller
Model
feature 2
feature 1
tangling
scattering
A real-world metaphor
A real-world metaphor
http://xkcd.com/657/
Mapping SoC to other principles
SEPARATION OF CONCERNS
Mapping SoC to other principles
SEPARATION OF CONCERNS
LOW SCATTERING LOW TANGLING
Mapping SoC to other principles
SEPARATION OF CONCERNS
Low coupling[Stev’74]
Information hiding[Parn’72]
DRY[Hunt’99]
LOW SCATTERING LOW TANGLING
Mapping SoC to other principles
SEPARATION OF CONCERNS
Low coupling[Stev’74]
High cohesion[Stev’74]
Information hiding[Parn’72]
DRY[Hunt’99]
Single responsibility[Mart’02]
Common closure[Mart’02]
LOW SCATTERING LOW TANGLING
Measuring SoC
• Concern location + concern-oriented metrics
– Trialed at Motorola [Simm’06]
• Static analysis + Cohesion and Coupling
– Issues: coupling vs. DRY, “uncohesive” java.util
• Repository mining for change-sets:
http://swerl.tudelft.nl/bin/view/Main/TestHistory
clas
ses
Summary
• Module system is a tool, not the goal
• No “silver-bullet” modularization
• Restructuring layers into features is viable
• SoC – the root of all principles
References[Parn’72] Parnas, D.L. (1972). On the criteria to be used in decomposing systems into modules. Communications of the
ACM, 15(12).
[Moad’90] Moad, J. (1990). "Maintaining the competitive edge". Datamation 61-62, 64, 66.
[Erli’00] Erlikh, L. (2000). "Leveraging legacy system dollars for E-business". (IEEE) IT Pro, May/June 2000.
[Nose’90] Nosek, J. & Palvia, P. (1990). "Software maintenance management: changes in the last decade". Journal of Software Maintenance: Research and Practice 2 (3).
[Mart’11] Martin, R.C. (2011). http://blog.8thlight.com/uncle-bob/2011/09/30/Screaming-Architecture.html
[Dijk’74 ] Dijkstra, E. W. (1974). On the role of scientific thought. Selected Writings on Computing: A Personal Perspective.
[Kicz’96] Kiczales, G., Irwin, J., Lamping, J., Loingtier, J., M., Lopes, C., V., Maeda, C. and Mendhekar, A. (1996). Aspect-oriented programming. ACM Computing Surveys, vol. 28.
[Tarr’99] Tarr, P., Ossher, H., Harrison, W. and Sutton, S. M. (1999). N degrees of separation: multi-dimensional separation of concerns. In ICSE’99: Proceedings of the 21st international conference on Software engineering.
[Leto’86] Letovsky, S. and Soloway, E. (1986). Delocalized Plans and Program Comprehension. IEEE Software, 3(3).
[Ruga’95] Rugaber, S., Stirewalt, K. and Wills, L. M. (1995). The interleaving problem in program understanding. In WCRE’95: Proceedings of the 2nd Working Conference on Reverse Engineering.
[Simm’06] Simmons, S., Edwards, D., Wilde, N., Homan, J. and Groble, M. (2006). Industrial tools for the feature location problem: an exploratory study. Journal of Software Maintenance and Evolution Research and Practice, 18(6).
[Mart’02] Martin, R. C. (2002). Agile Software Development, Principles, Patterns, and Practices. Prentice Hall.
[Stev’74] Stevens, W. P., Myers, G. J. and Constantine, L. L. (1974). Structured Design. (E. Yourdon, Ed.)IBM Systems Journal, 13(2).
[Hunt’99] Hunt, A., & Thomas, D. (1999). The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley Professional.
(Many of these papers are available online through Google Scholar, Citeseerx and the authors’ websites)
Feature tracer [backup]
Part III
Paradoxes of API Design
Modularity is relative to change
• API are like stars (paradox 19)
• Designing a universe
• Distributed development (paradox 2)
• Can't know all your users
• Envisioning them via use-cases
• Sustaining (paradox 3)
• One try to get API right
How to anticipate future changes?
• Client vs. Provider APIs (paradox 9)
• Open spaces
• Fixed points
• Stable API evolves
• Backward compatibility (paradox 6)
• Beware of API-less APIs (paradox 8)
• Mediawiki experience
• Loyal customer
Logical vs. Physical Design
• Design oriented on class relationship
• UML, specifications
• Packaging into JARs ignored (paradox 17)
• Influences deployment (paradox 7)
• Defines APIs
• Good common ground (paradox 11)
• Improves your design
A „weight“ of a module
• Environment for a module
• Modules don't live in vacuum
• Expressed by dependencies (paradox 17)
• Weight
• Number & scope of outgoing dependencies
• Less is more (paradox 19)
• Single responsibility principle
Use vs. Re-use
• Kirk Knoernchild
• Monolithic API is easier to use
• Modular API is easier to re-use
• Blackbox pattern (paradox 18)
• OSGi Capabilities
• Good tooling
• Wizards
SOLID APIs
• Single responsibility principle
• Meaning of modifiers (paradox 15)
• Client vs. provider APIs (paradox 9)
• Lightweight API modules
• Open/closed principle
• OK for provider APIs
• Disastrous for client APIs
• Proliferation of instanceof in user code
• Alternative behavior (paradox 16)
SOLID APIs II
• Liskov substitution principle
• AWT Frame extends Component!
• Don't expose deep hierarchies
• Use delegation rather than inheritance
• Client API should be in final classes
• 1:N factory methods
SOLID APIs III
• Interface segregation principle
• Lookup & discover
• OSGi declarative services
• Dependency inversion principle
• Code against interfaces, not implementations
• Does not imply classes are bad (paradox 9)
• Don't fear (injectable) singletons (paradox 14)
API in User Eyes
• Clueless users (paradox 1)
• Have always something else to do
• Evaluation of an API (paradox 4)
• Coolness
• Time to market
• Total cost of ownership
Collaboration
• Maintenance (paradox 10)
• Rely on patches
• Accepting unacceptable (paradox 13)
• Beauty (paradox 5)
• One writer and dozens of users
• Sacrifice the writer
Summary
• http://paradoxes.apidesign.org
Recommended