Embed Size (px)
Citation preview
Benefits and Drawbacks of Software
Reference Architectures:
A Case Study
IST Journal First, ESEM 2017. Toronto, November 9th 2017
Silverio Martínez-Fernández, Claudia Ayala, Xavier Franch, Helena Marques
Fraunhofer IESE, UPC-BarcelonaTech, everis
@silveriomf
Definitions
Software Architecture: the set of structures needed to
reason about the system
Software Reference Architecture (SRA):
a software architecture where the structures and
respective elements and relations provide templates for
concrete architectures in a particular domain
2
Reference Architectures
Software Reference Architectures (SRA) help design concrete architectures:
• Software elements, guidelines, documentation... for designing systems in a domain
• Reusable architecture knowledge
• Starting point to specialize for project context
• Popular in industry
With the goal of:
• Providing guidelines and inspiration for the design of systems
• Achieving standardization
3
Examples of Software Reference Architectures
Technological domain Business domain
For many organizations For a single organization
100+ Software reference architectures in scientific literature (e.g., service-oriented systems, automotive software)
75+ Software reference architecting practices in scientific literature (e.g., design methods, conformance checking)
4
Motivation
Several potential benefits of using reference architectures have been reported, but
• Most of them are not supported by empirical evidence
• The perspective of academics is not always inline with industrial practice
Our research goal is:
To gather evidence regarding the benefits and drawbacks of SRAs in a company from the
perspective of different stakeholders involved in their design and usage
We conducted a case study at everis, a multinational software company
5
Case Study: Methodology and Results
6
The Context at everis
7
Problem in everis
To provide a solution to the challenges that
everis faced in SRA projects:
• To assess if an SRA is a good approach for
software development in a client organization
• To align current SRA projects to the client
organization’s goals
How can an organization get corporate evidence that is useful for the SRA engineering?
8
Research Questions
RQ1 What are the main benefits of SRAs in the context of the client organizations of everis?
RQ1.1 What are the main similarities and differences among stakeholders’ perception of such benefits?
RQ2 What are the main drawbacks of SRAs in the context of the client organizations of everis?
RQ2.1 What are the main similarities and differences among stakeholders’ perception of such drawbacks?
RQ2.2 What are the potential improvements that stakeholders would be willing to perform to overcome the
drawbacks of SRAs?
9
Design and Sampling
Qualitative and Exploratory Case Study
Target: everis’ SRA projects
Selection of 9 SRA projects based on feasibility to
contact at least with one of each role
28 stakeholders
10
Sampling
11
Id org.
SRA project
composition Main domain Applications based on the SRAApprox. effort
(hours)SA AD AB
A 2 1 3 Industry Web-based applications to allow vendors to update
information about clients in a department store.
≈5,000
B 3 6 1 Banking Multi-platform applications that are fast, satisfy
practices of the market and support transaction
processing.
≈97,000
C 6 3 1 Banking Multi-platform applications of a bank with improved
usability.
≈37,000
D 1 10 3 Insurance Applications that satisfy internal requests for
proposals.
≈29,000
E 2 1 1 Public sector Java web applications, with flexible front-end,
integration and batch processes.
≈6,500
F 1 3 2 Public sector Web-based applications for the different departments
of a public administration.
≈20,000
G 2 2 1 Public sector Applications with enhanced reusability and reduced
development costs.
≈4,500
H 3 2 1 Insurance Applications integrated with the services of an
insurance company.
≈4,000
I 1 3 1 Public sector Applications that include the business processes of a
utility organization.
≈6,500
Data Collection and Instruments
Gathering data:
Interview transcripts
Online questionnaires: Lime Survey
Other sources: SRA project card, documentation, metrics
about invested effort, informal meetings
Research data: Instruments, data collected, and analysis
https://data.mendeley.com/datasets/y5r8rzf3xg/243
12
Data Analysis
Processing the open questions from the interviews: a tailored
thematic analysis (grouping, coding, categories, discussions
among researchers)
Online questionnaires: automatically reported by the tool
Cluster analysis (simple k-means) to identify groups
13
14
RQ1 Results: SRA Benefits
RQ1.1 Results: Stakeholders Perceptions
15
16
RQ2 Results: SRA Drawbacks
17
RQ2.1 Results: Stakeholders Perceptions
RQ2.2 Results: Improvements
18
Code Category Representative quotes from software architects # SA # AD # AB %
Total
Imp-A Add functionality or modules to the
SRA
“Our aim is to foster a visual plugin that makes the
development easier and automates the
development of components for the application
builders more” [C].6 4 8 64%
Imp-B Technology change “We are limited to JSF; migrating to another
framework, like Sencha, would allow offering
presentation not only for web browsers, but also
for mobile devices” [G].5 2 1 29%
Imp-C Simplify modules “There is a module that is more complete than
what is really asked for. Thus, it is not aligned with
the client needs and therefore it should be
simplified” [E].1 2 0 11%
Imp-D Add new practices or guidelines to
the SRA
“The main point is to move to continuous
integration” [H].2 0 0 7%
Imp-E Migrate from legacy applications “There are very old software elements that we
have inherited. It would be good to update them
because you do not know them well since they are
black boxes” [B].1 0 0 4%
Discussions
19
Theoretical benefits vs. this case study
Benefits Diagnostic %
Standardization (B1) ± 33%
Facilitation (B2) √ 68%
Reuse (B3) √ 82%
Risk reduction (B4) ± 33%
Enhanced quality (B5) ± 82%
Interoperability (B6) ° -
Knowledge repository (B7) √* 67%
Improved communication (B8) ±* 11%
Elaboration of mission, vision and strategy (B9) × 7%
Best practices (B10) ° -
Novel design solutions (B11) ± 53%
20
The second column shows the extent to which the results from our study support (√), partially support or
help to understand (±), do not explicitly mention (°), or refute (×) these benefits.
Benefits of using the everis reference model (instead of SRAs), indicated by (*)
Theoretical drawbacks vs. this case study
21
Drawbacks Diagnostic %
Initial investment (D1) ± 4%
Inefficient instantiation (D2) ± 11%
Too abstract (D3) ° -
Term confusion (D4) ± 43%
Bad documentation (D5) × 4%
Bad quality (D6) ± 7%
Limiting (D7) ± 28%
Learning curve new 63%
Dependency on the SRA new 22%
Complexity new 14%
Benefits and drawbacks by application domain
Banking domain:
Effort invested
Maintaintenance costs (Ben-B)
Learning curve (Dra-A)
Public sector:
Standardization (Ben-F)
22
All domains except public
sector:
Easier development (Ben-C)
Lastest technologies (Ben-D)
Limitations (Dra-B)
Dependency in the SRA (Dra-C)
Conclusions
A case study was conducted to analyze how SRAs are perceived by industrial
practitioners from nine client organizations of everis
Supported already known benefits, mainly cost savings in the development and evolution of
applications, and facilitation of the design of concrete software architectures
Revealed drawbacks: higher learning curve, higher time-to-market for applications when
their dependencies on the SRA are managed inappropriately, and high complexity of SRAs
when they do not properly aim to facilitate the development of applications
Analysis of different perceptions of stakeholders
23
Future Work
Study: a meta-analysis
Population: empirical studies about SRAs benefits and drawbacks
Respondents/Sampling: 5 primary studies
Instruments: systematic review for selection, structured synthesis method for
agreggation, and data analysis for studying the context
24
ESEM 2015
