Upload
ikmal-syifai
View
221
Download
0
Embed Size (px)
Citation preview
7/27/2019 Software and Large Software Development
1/55
LARGE SCALE SOFTWARE
DEVELOPMENT
IF3250 PROYEK PERANGKAT LUNAK
4 SKS
Sem I 2013/2014
7/27/2019 Software and Large Software Development
2/55
Large Systems
Here in Syracuse at least two very large softwaresystems have been successfully implemented:
Over The Horizon Radar (OTHR) contained about3,000,000 lines of code, written mostly in FORTRAN withsome C and some assembly language as well.
BSY-2 submarine battle management system software is
several times larger than OTHR, written mostly in Ada.
A 5 million line system would require something like:
effortofyearspersonyeardays
xdaylines
lines947
/240
1
/22
000,000,5=
7/27/2019 Software and Large Software Development
3/55
Large Systems (2)
So, to complete the project in 2.5 years would requirethe services of at least 380 well trained softwaredevelopers.
Two points are almost self evident:
a system this large must be partitioned into many rela-tivelysmall, nearly independent components in order to get it towork
it would be much better not to create such a large sys-temat all, but rather, to build most of it from reused swcomponents, reserving new code for new requirements.
7/27/2019 Software and Large Software Development
4/55
SW Product Model
Architecture defines product components and allocates processing to
them
defines external product behavior
Requirements Specification describes what constitutes correct operation
it is the basis for testing and evaluation
Product Specification
defines an architecture for the system describes software design and implementation
specifies a software build process
7/27/2019 Software and Large Software Development
5/55
SW Product Model (2)
Test Plan defines procedures for unit, integration, validation, qualification, and regression
testing
qualification test procedures are emphasized
Prototype Code
verifies design for critical processing, analyzes implementation problems as theyarise
Product Code
Code for each component of the product, implemented as software modules.
test stub attached to each module, used to establish basic software cycling andnearly correct operation
Test Code test drivers for unit, integration, and qualification tests
Test Report
7/27/2019 Software and Large Software Development
6/55
Large Scale (LS) Software Development
Complexity in Software High LOC, High amount of Data Acces, High computation
elements, etc
Complexity in Requirements High Requirements Size, Requirements Changes, conflicting
requirements, etc
Complexity in location differences
Complexity in platform differences
Complexity in politics Different rules, policies
Complexity in Resource Allocations
7/27/2019 Software and Large Software Development
7/55
Coordination in LS SW Development
The root cause is work coordination between team
developers
intra- and inter- team coordination
Major factors are Communication
Capacity
Cooperation
7/27/2019 Software and Large Software Development
8/55
What should we be aware of?
Software Change
Software Configuration Management (SCM)
Change Control
7/27/2019 Software and Large Software Development
9/55
Software Change Management
Environment Changes
Computer Hardware
Competition
Business Processes
External Requirements (Y2k, Euro,)
New Standards (XML, MP3, e-Speak,)
All Software Demands Change Incremental Development
Maintenance (Corrective, Perfective, Adaptive)
7/27/2019 Software and Large Software Development
10/55
Source Control Management
Manage Constant Software Change
Improve Debugging Productivity
Trace Defect Reports to Correct Versions of Source Code
Understand Change History Reproducibility
Build (Compile) Software Accurately, Repeatably
Grow Project Size
Avoid Over-writing and Losing Changes Concurrent Development (Improve TTM)
Coordinate Work with Development Policies
7/27/2019 Software and Large Software Development
11/55
Versioning
Revision / Version - Single copy of a source file as it
appeared at given point in its change history
(checkpoint)
File / Element - Complete change history; allversions
Branch - Set of successive versions; parallel to other
branches Delta - Change from one file version to the next
7/27/2019 Software and Large Software Development
12/55
Version Tree
7/27/2019 Software and Large Software Development
13/55
Multiple File Configuration Management
Variant - File based on another, co-exists with
original
Derived Object - File automatically generated from
source files Change Set - Set of versions across multiple files
making up a single logical change (e.g., feature or
fix) Baseline - Consistent set of versions across all files
7/27/2019 Software and Large Software Development
14/55
Baseline
fubar.c foo.c bar.c
7/27/2019 Software and Large Software Development
15/55
Coordinating Work
Problem: Multiple people working on same file atthe same time can result in lost work.
Solution 1: Serialize work with file locking.
Solution 2: Work in isolation (branch) andmerge.
Branching and merging scales much better.
Merging can be semi-automated using 3-waymerge algorithms.
7/27/2019 Software and Large Software Development
16/55
Merging - Version Tree
0
1
2
1
0
Common
Ancestor
ConflictingChanges
3
Merge
Result
7/27/2019 Software and Large Software Development
17/55
Formal definition (IEEE)
SCM is the process of identifying and defining the
items in the system, controlling the change of these
items throughout their life cycle, recording and
reporting the status of items and change requests
and verifying the completeness and correctness of
the items.
Related Terms: Source Control, Version
Control
7/27/2019 Software and Large Software Development
18/55
Managing Change
Change Control -Formal processes for tracking andapproval of software modifications.
Managed with SCM and defect tracking tools.
Usual Policy Areas: Documentation - Change request / defect report
filed.
Approval - Change request reviewed by releasemanagement or technical board.
Quality Requirements - Code reviewed, regressiontests passed.
Policies often change by release stage and in responseto problems.
7/27/2019 Software and Large Software Development
19/55
Development Process
Key Components of Development Process:
Software Configuration Management
Change Control Quality Control
7/27/2019 Software and Large Software Development
20/55
Principles of Large-Scale Software
Development
Reproducibility
Policy Enforcement versus Policy Auditing
Process, Process, Process
A process must be defined to be improved
Automation
A process must be automated to be repeatable
Continuous Functional Integration and Test
7/27/2019 Software and Large Software Development
21/55
Software Integration
Integration: Combining multiple features orcomponents developed in isolation.
Large projects are subject to constant change, soisolation on a stable base is necessary.
Continuous Functional Integration and Test (CFIT)
Changes integrated throughout development.
Changes not allowed to regress functionality orquality.
7/27/2019 Software and Large Software Development
22/55
HP-UX Development
220,000+ Source Files
~2.5 Million Versions
52,000+ Derived Files (Single Nightly Build)
900+ User Accounts
More Than 1 Terabyte of Disk Space
7/27/2019 Software and Large Software Development
23/55
HP-UX Concurrent Development
Normally 2-3 Releases Under Development
Several Releases Under Maintenance
Heavy Use of Branching and Merging
7/27/2019 Software and Large Software Development
24/55
HP-UX Multisite Development
Four Primary Development Sites
Three in USA (Three Time Zones)
One in India
One USA Site Hosts Three Logical Sites Partner Companies
Several Secondary Sites
7/27/2019 Software and Large Software Development
25/55
Tiered Software Integration Model
1. Developer Task - Edit, Compile, Test
2. Submit - Batch Changes, Acceptance Test3. Baseline - Nightly Build, Regression Test
4. Integration - Monthly Packaging, Partner Test
5. Release - Semi-annual Delivery, Production Use
7/27/2019 Software and Large Software Development
26/55
Integration Levels
Build
Submit
Baseline
Integration Cycle
Release
Modifications
7/27/2019 Software and Large Software Development
27/55
Process Evolution
Continuous Process Improvement
Root Cause Analysis (RCA)
Capability Maturity Model (CMM)1. Initial
2. Repeatable
3. Defined
4. Managed5. Optimizing
7/27/2019 Software and Large Software Development
28/55
Tools
7/27/2019 Software and Large Software Development
29/55
Commercial (license cost per concurrent user, approx 2-6Keach) Rational/IBM XDE Java Edition 2002/XDE .NET Edition 2002
Borlands Together (acquired TogetherSoft)
Telelogics System Architect 10 (acquired Popkin Software) Microsofts Enterprise Architect 2002
Ilogixs Rhapsody
Gentlewares Poseidon for UML
Free Tigris ArgoUML (Open Source) EclipseUML (Open Source)
StarUML (Open Source)
UML-supported CASE tools
7/27/2019 Software and Large Software Development
30/55
System Architect 10
Telelogic (acquired Popkin Software in 2005)
Types of modelling supported: Business Process Modelling Object/Component Modelling
Data Modelling Structured Analysis and DesignInterface: Drawing Workspace Browser - hierarchical navigation (of diagrams and definitions) Definition dialogues Matrix EditorImport/Export Formats Comma Separated Vale (csv), XML
7/27/2019 Software and Large Software Development
31/55
System Architect 10
Reporting: Proprietary, MS Word templates, HTML Generation
Object Modelling UML and old methods (Booch, OMT, etc.)
Code Generation C++ (+ reverse engineering), Java (+ reverse engineering), Visual
Basic, CORBA IDL, Javascript
Schema Generation
Data Defintion Language of SQL, e.g. Access, ORacle, etc.
GUI screen painter
7/27/2019 Software and Large Software Development
32/55
Together
comprehensive, multi-language modeling platform forsoftware practitioners who design, develop, and/orcommunicate software architectures
Two versions Together 2006 Release 2 for Eclipse (3.2)and Together 2006 for Visual Studio
support for UML 2.0, OCL, XMI, Design Patterns, Java5, and CORBA/IDL
Language Support: Eclipse Version Java and C++ Visual Studio Version C# and VB.NET
Database Support SQL Server, Oracle, DB2 andSybase
7/27/2019 Software and Large Software Development
33/55
7/27/2019 Software and Large Software Development
34/55
Complex Softwarehttp://www.intelligent-files.com/complex-software.html
Almost everything, around and about applicationsoftware undergoing change, conspires to move iton from being justsoftware, to being complexsoftware
Complex software arises from a culmination ofmany factors.
The simplest measure of software complexityis via an
algorithm called McCabe's Cyclomatic Complexity Complex software can be defined through a
summation of distinct measures.
7/27/2019 Software and Large Software Development
35/55
Bringing complex software to ground level, this is what you feel, as adeveloper working with complex software: complex software will make it hard to determine where to make a
change to add a feature or fix a bug.
complex software will cause what is called cascading changes. A simple
change suddenly requires you to change ten, twenty, even thirtysoftware files.
complex software when changed will produced many unintended sideeffects, most of these being unwanted, even destructive.
complex software will contain much duplication. This means, that formany changes, you will need to repeat yourself, sometimes many times
over. complex software is hard to maitain. Data patches, manual releases, the
lack of automated tests all add up to disempower IT maintenance teams.
Complex Softwarehttp://www.intelligent-files.com/complex-software.html
7/27/2019 Software and Large Software Development
36/55
Contibuting factors
the developers, their psychology, fear, communication skills, desirefor feedback.
the business. its risk appetite, rewards structure, its ability to recruit& retain.
the market. socio-economics, technology change, regulatory climate,customer mobility
the project its leadership, sponsors, its agility & budget, its pace ofdevelopment
the competition systems, their technology & responsiveness, their
differentiators the customertheir loyalty, their need for customization, their financial
disposition
Complex Softwarehttp://www.intelligent-files.com/complex-software.html
7/27/2019 Software and Large Software Development
37/55
Ultra Large Scale (ULS) System
Ultra-large-scale system (ULSS) is a term used in fields includingComputer Science, Software Engineering and Systems Engineering torefer to software intensive systems with unprecedented amounts ofhardware, lines of source code, numbers of users, and volumes ofdata.
The scale of these systems gives rise to many problems:they will be developed and used by many stakeholders acrossmultiple organizations, often with conflicting purposes and needs;they will be constructed from heterogeneous parts with complexdependencies and emergent properties; they will be continuouslyevolving; and software, hardware and human failures will be the
norm, not the exception.
(http://en.wikipedia.org/wiki/Ultra-large-scale_systems)
http://en.wikipedia.org/wiki/Computer_Sciencehttp://en.wikipedia.org/wiki/Software_Engineeringhttp://en.wikipedia.org/wiki/Systems_Engineeringhttp://en.wikipedia.org/wiki/Systems_Engineeringhttp://en.wikipedia.org/wiki/Software_Engineeringhttp://en.wikipedia.org/wiki/Computer_Science7/27/2019 Software and Large Software Development
38/55
Ultra Large Scale SoftwareDevelopment
7/27/2019 Software and Large Software Development
39/55
The scale and complexityhttp://www.sei.cmu.edu/uls/
The scale and complexity of systems is increasing dramatically. Ultra-large-scale (ULS) systems are systems of unprecedented scale
in some of these dimensions: lines of code
amount of data stored, accessed, manipulated, and refined
number of connections and interdependencies number of hardware elements
number of computational elements
number of system purposes and user perception of these purposes
number of routine processes, interactions, and emergent behaviors
number of (overlapping) policy domains and enforceable mechanisms number of people involved in some way
ULS S Ch i i
7/27/2019 Software and Large Software Development
40/55
ULS System CharacteristicsUltra-Large-Scale Systems; The Software Challenge of the Future
(Software Engineering Institute Carnegie Mellon)
Decentralization The scale of ULS systems means that they will necessarily be
decentralized in a variety of waysdecentralized data, development,evolution, and operational control.
Inherently conflicting, unknowable, and diverse requirements
ULS systems will be developed and used by a wide variety ofstakeholders with unavoidably different, conflicting, complex, andchanging needs.
Continuous evolution and deployment:
There will be an increasing need to integrate new capabilities into a ULSsystem while it is operating. New and different capabilities will be
deployed, and unused capabilities will be dropped; the system will beevolving not in phases, but continuously.
7/27/2019 Software and Large Software Development
41/55
ULS System Characteristics (2)
Heterogeneous, inconsistent, and changing elements: A ULS system will not be constructed from uniform parts: there will
be some misfits, especially as the system is extended andrepaired.
Erosion of the people/system boundary
People will not just be users of a ULS system; they will beelements of the system, affecting its overall emergent behavior.
Normal failures
Software and hardware failures will be the norm rather than theexception.
New paradigms for acquisition and policy The acquisition of a ULS system will be simultaneous with the
operation of the system and require new methods for control.
ULS Ch ll
7/27/2019 Software and Large Software Development
42/55
ULS ChallengesUltra-Large-Scale Systems; The Software Challenge of the Future
(Software Engineering Institute Carnegie Mellon)
Design and Evolution
Orchestration and Control
Monitoring and Assessment
7/27/2019 Software and Large Software Development
43/55
7/27/2019 Software and Large Software Development
44/55
ULS Challenges (3)
Orchestration and Control By orchestration we mean the set of activities needed to make the
elements of a ULS system work in reasonable harmony to ensurecontinuous satisfaction of the mission objectives. Orchestrationinvolves management and administration but at a scale well
beyond that of traditional, centralized, relatively fine-grainedcontrols. Orchestration requires a combination of up-front design,overall policy promulgation and enforcement, and real-timeadjustment of operating parameters.
Orchestrating a ULS system requires supportinginterdependencies and controlling the consequences of local
actions with respect to their effect on the emergent whole, eventhough each part of a system might be acting to maximize itslocal utility.
7/27/2019 Software and Large Software Development
45/55
ULS Challenges (4)
To succeed in developing and operating ULS systems,
new knowledge, technologies, and methods in the
following areas are needed:
online modification
maintenance of quality of service
creation and execution of policies and rules
adaptation to users and contexts
enabling of user-controlled orchestration
7/27/2019 Software and Large Software Development
46/55
ULS Challenges (5)
Monitoring and assessment
The effectiveness of ULS system design, evolution, and
orchestration has to be evaluated. There must be an
ability to monitor and assess ULS system state,behavior, and overall health and well being.
The criteria for success or overall health are different
for ULS systems than for smaller systems designed to
accomplish a task that does not change as the system isused.
D i i hi h ULS S i
7/27/2019 Software and Large Software Development
47/55
Domains in which ULS Systems are emerging
http://www.sei.cmu.edu/uls/
Defense
The Northrop report argued that "the U.S. Department of Defense (DoD) has a goal of
information dominance this goal depends on increasingly complex systems characterized by
thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through
heterogeneous wired and wireless networks. These systems will push far beyond the size of
today's systems by every measure They will be ultra-large-scale systems."[1]
Financial trading
Following the flash crash, Cliff and Northrop[2] have argued "The very high degree of
interconnectedness in the global markets means that entire trading systems, implemented and
managed separately by independent organizations, can rightfully be considered as significant
constituent entities in the larger global super-system. The sheer number of human agents
and computer systems connected within the global financial-markets system-of-systems is so
large that it is an instance of an ultra-large-scale system, and that largeness-of-scale hassignificant effects on the nature of the system". [2]
http://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/2010_Flash_Crashhttp://en.wikipedia.org/wiki/Dave_Cliff_(professor)http://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/Dave_Cliff_(professor)http://en.wikipedia.org/wiki/2010_Flash_Crashhttp://en.wikipedia.org/wiki/Ultra-large-scale_systems7/27/2019 Software and Large Software Development
48/55
Domains.... (2)
Healthcare
Kevin Sullivan has stated that the US healthcare system is "clearly anultra-large-scale system"[7] and that building national scale cyber-infrastructure for healthcare "demands not just a rigorous, modernsoftware and systems engineering effort, but an approach at the
cutting edge of our understanding of information processing systemsand their development and deployment in complex socio-technicalenvironments".[7]
Others
Other domains said to be seeing the rise of ultra-large-scalesystems include government, transport systems (for example airtraffic control systems), energy distribution systems (for examplesmart grids) and large enterprises.
http://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/Ultra-large-scale_systemshttp://en.wikipedia.org/wiki/Ultra-large-scale_systems7/27/2019 Software and Large Software Development
49/55
Managing Complexity
Software engineers have devised a number of fundamental tools of the trade forhelping to manage complexity [1]:
Decomposition: The most basic technique for tackling large problems is to divide them
into smaller, more manageable chunks, each of which can then be dealt with in relative
isolation (note the nearly decomposable subsystems in Figure 1). Decomposition helps
tackle complexity because it limits the designers scope.
Abstraction: The process of defining a simplified model of the system that emphasizes
some of the details or properties, while suppressing others. Again, this works because it
limits the designers scope of interest at a given time.
Organization: The process of defining and managing the interrelationships between the
various problem-solving components (note the subsystem and interaction links of Figure 1).
The ability to specify and enact organizational relationships helps designers tackle
complexity by: enabling a number of basic components to be grouped together andtreated as a higher-level unit of analysis, and providing a means of describing the high-
level relationships between various units.
[1] Booch, G. Object-Oriented Analysis and Design with Applications. Addison Wesley, 1994.
Very Large Scale Software
7/27/2019 Software and Large Software Development
50/55
Very Large-Scale Software
Engineering (VLSSE)
Architectural specification (system, services (data,
protocols, processes), team/organization.
Configuration management and version control
Process automationbuilds, regression tests, etc.
Reliance on informalisms for development
(Open Source Software Development and Very Large-Scale SoftwareEngineering (Walt Scacchi Institute for Software Research University of
California, Irvine Irvine, CA, 92697-3425 USA 19 July 2005))
7/27/2019 Software and Large Software Development
51/55
VLSSE (2)
Information density vs. knowledge coverage
VLSEE needs for ASB to consider: Test-beds for full-size system mock-ups (probably built from
OSS and Computer Game components)
Visualization aidsarchitecture, configuration, socio-technical networks
Empirical knowledge about existing VLSEE practices for bothOSS and CSS
Computer-supported cooperative organizational learningenvironments (CSCOLE) supporting long-term organizationaltransformation.
7/27/2019 Software and Large Software Development
52/55
VLSSE - The Process
Configuration management and version control
Architectural specification and maintenance
Collaboration, leadership, control and conflict
management Community development and support
Software source code and artifact data mining
G d B h d i b t ki bi
7/27/2019 Software and Large Software Development
53/55
Grady Boochs advices about making big
software development projects a success
The fundamentals never go out of style, never forget the fourfundamentals: Create crisp and resilient abstractions.
Maintain a good separation of concerns.
Create a balanced distribution of responsibilities.
Focus on simplicity. You need a regular rhythm of releases.
"Establishing that rhythm provides predictability and sustainability."Regularity lets everyone plan on what functionality or features will beinjected into the next round, regardless of release frequency.
Focus upon growing executable architectures. IT managers need to govern around the architectural decisions rather
than raw, running, naked code. "The code is the truth," Booch says. "Butthe code is not the whole truth.
Grady Boochs advices
http://en.wikipedia.org/wiki/Abstraction_(computer_science)http://en.wikipedia.org/wiki/Abstraction_(computer_science)7/27/2019 Software and Large Software Development
54/55
Grady Booch s advices...(http://www.cio.com/article/373215/5_Things_Grady_Booch_Has_Learned_About_C
omplex_Software_Systems?page=2&taxonomyId=3020)
Create social structures that encourage innovation while still preserving
predictability.
Teamwork is an essential ingredient in any large software project, so
companies should give attention to creating relationships that work.
Businesses need predictability (when will this software really ship?) and
they also want innovation (make it do something cool!).
Building a social structure to support both those goals isn't necessarily
easy, but the successful projects find a balance.
Keeping developers out of "blasted meetings" so they can get things
done. Have fun.
Successful projects come from teams that are jazzed about what they're
doing. "Most people want to build beautiful, elegant things,"
7/27/2019 Software and Large Software Development
55/55
Tugas
Bagi kelompok @5 orang, dikumpul (Hardcopy) Selasa(3/9/2013)
Cari paper terkait large scale atau complex software,pelajari, dan siapkan presentasi singkatnya (utk Selasa)
Contoh: Coordination in Large-Scale Software Team
Survey answered by 775 Microsoft software engineers
Parallel Changes in Large Scale Software Development: AnObservational Study
Coodination in Large-Scale Software Development: Helpful andUnhelpful Behaviors
Defining Execution Viewpoints for a Large and ComplexSoftware-Intensive System