Gaining Intellectual Control of

Software Development*

Results of an NSF Software Engineering Research Strategies Workshop

Barry Boehm, USC

boehm@usc.edu

http://sunset.usc.edu/~boehm/

September 27, 1999

* includes software evolution, legacy software adaptation

29/27/99

NSF Participants:

– Frank Anger– Michael Evangelist

Remote Participants:

– Fred Brooks– Jim Gray– Martin Griss– Daniel Jackson– David Luckham– Tony Wasserman– William Wulf*

– Fran Allen – Bob Balzer– Vic Basili*– Barry Boehm**– Lori Clarke– Rich DeMillo– Peter Freeman– John Goodenough– Susan Graham*– Beki Grinter– George Heineman– Jim Horning

* - Program Committee** - Workshop Chair

– Dick Kemmerer– John Knight– Nancy Leveson*– Neno Medvidovic*– Lee Osterweil*– Dewayne Perry*– Bill Scherlis*– Mary Shaw– Vicky Stavridou, – Kevin Sullivan – Richard Taylor*– Will Tracz– Stu Zweben

Workshop Participants:

Acknowledgements

39/27/99

Outline: Workshop Results

– Scope and Context

– Nature and Role of Software Engineering

– Software Engineering Research Strategies

– Conclusions

49/27/99

Scope and Context

• PITAC Report– Need significant boost in IT research

– Software is too fragile and unpredictable

– Software research a top priority

• IT² Report– Structure of proposed federal IT research initiative

• 1998 NSF Software Research Program Workshop– Set goal of achieving “no-surprise” software

– Identified research program components– Product, process, domain, empirical research– Transitioning research into practice

• This Workshop– Clarify nature and role of software engineering

– Identify software engineering research strategies

59/27/99

Role of Software Engineering in IT Research and Systems

Architectures,Composition

Frameworks &Principles

Great ITComponents

Great SWEngineering

Great Systems

+ =

Networks

OS, DBMS,Middleware

AI, Agents

User Applications

User InterfacesDevelopmentStakeholders

Operational Stakeholders

System Definition,Composition,

Verification, andEvolution Processes

Modeling and Analysis

HCI & Collaboration

User Applications

Info Distribution &Management

Connectivity &Information Access

Quality ofService

Technologies

Test & Verification;Usage Evaluation

69/27/99

• Increased complexity– Everything connected– Opportunities for chaos (agents)– Systems of systems

• Decreased control of content– Infrastructure– COTS components

• Faster change– Time-to-market pressures– Marry in haste; no leisure to repent– Adapt or die (e-commerce)

• Fantastic opportunities– Personal, corporate, national, global

Future Software Trends

79/27/99

Outline: Workshop Results

• Scope and Context

• Nature and Role of Software Engineering

• Software Engineering Research Strategies

• Conclusions

89/27/99

SE Research Strategies: Workshop Approach

SE-Intensive Challenge Areas• Empowering people and groups• Weaving the new information fabric

Example challenge problems;SE research needs

SE research strategiesCritical success factorsfor SE research

Trends in IT demand,technology

PITAC challengesand IT goals

99/27/99

• User Programming– Dewayne Perry, Lucent

• Empowered Teams

• Lifelong Learning

• Embedded Medical Systems

Empowering People and Groups: Workshop Examples

109/27/99

• Crisis Management–William Scherlis, CMU

• Air Traffic Control

• On-Demand Organizations–Leon Osterweil, UMass

• Medical Informatics

Weaving the New Information Fabric: Workshop Examples

119/27/99

Example Research and Technology Areas

• Software Architectures– Nenad Medvidovic, USC

• Connectivity and Information Access– Richard Taylor, UCI

• Modeling and Analysis– Victor Basili, UMD

129/27/99

Essential

Strong

Moderate

None

Degree of Dependence

Integration of Technology Elements

Software Engineering Technologies

Software Engineering Technologies, Mission Challenges, and Underlying Science

UnderlyingScience

Weaving the New Information Fabric

Empowering Peopleand Groups

Process Technologies

System Definition

Architecture

Composition

Test & VerificationUsage Evaluation & Evolution

Process Modeling & ManagementProduct Technologies

HCI & Collaboration

User Domain Componentry

Connectivity & Info. Access

Info. Distribution & Mgmt. Quality of Service Technologies

High Assurance

Massive Scalability

Change ResilienceModeling & Analysis Technologies

Domain Modeling

Software Economics Modeling

Quality of Service Modeling

User P

rogr

amming

Empo

wered

Tea

ms

Life

long

Lea

rnin

gEm

bedd

ed

Medic

al Sy

stem

sCris

is Man

agem

ent

Air Tr

affic

Con

trol

Net-Cen

tric B

usines

sMed

ical In

form

atics

Compu

ter S

cienc

eDom

ain S

cienc

esBeh

avio

ral S

cienc

esEc

onom

ics

139/27/99

Agenda

• 8:30- 8:50 – Evangelist, Boehm: Overview• 8:50- 9:10 – Perry: User Programming• 9:10- 9:30 – Osterweil: On-Demand Organizations• 9:30- 9:40 – Break• 9:40-10:00 – Scherlis: Crisis Management• 10:00-10:20 – Medvidovic: Software Architecture• 10:20-10:40 – Taylor: Information Access and

Connectivity• 10:40-11:00 – Basili: Modeling and Analysis• 11:00-11:05 – Boehm: Conclusions• 11:05-12:00 – General Discussion

149/27/99

Essential

Strong

Moderate

None

Degree of Dependence

Integration of Technology Elements

Software Engineering Technologies

Software Engineering Technologies, Mission Challenges, and Underlying Science

UnderlyingScience

Weaving the New Information Fabric

Empowering Peopleand Groups

Process Technologies

System Definition

Architecture

Composition

Test & VerificationUsage Evaluation & Evolution

Process Modeling & ManagementProduct Technologies

HCI & Collaboration

User Domain Componentry

Connectivity & Info. Access

Info. Distribution & Mgmt. Quality of Service Technologies

High Assurance

Massive Scalability

Change ResilienceModeling & Analysis Technologies

Domain Modeling

Software Economics Modeling

Quality of Service Modeling

User P

rogr

amming

Empo

wered

Tea

ms

Life

long

Lea

rnin

gEm

bedd

ed

Medic

al Sy

stem

sCris

is Man

agem

ent

Air Tr

affic

Con

trol

Net-Cen

tric B

usines

sMed

ical In

form

atics

Compu

ter S

cienc

eDom

ain S

cienc

esBeh

avio

ral S

cienc

esEc

onom

ics

159/27/99

Conclusions: SE Research Needs

• Major needs for further SE science and technology (S&T)– Process, product, quality of service,

modeling and analysis– S&T integration across areas

• SE science base requires more than computer science– Need integration with domain sciences,

behavioral sciences, economics…– Need both specialist and interdisciplinary advances

• There is no single silver bullet for success– Major applications require many technologies– Need integrated SE/IT research programs

169/27/99

Conclusions: SE Research Strategies

• Future software trends create integrated SE/IT challenges– Increased complexity– Decreased control of content– Faster change– Fantastic opportunities

• Weaving new information fabric provides vision of solution

• Need integrated SE/IT research program to realize the vision

179/27/99

Backup charts

189/27/99

• Empowering People and Groups

• Weaving the New Information Fabric

SE-Intensive Grand Challenge Mission Areas

199/27/99

• Different counting rules

• Try counting software as Lines of Code in Service

= (#platforms) *

(#object LOC’s/platform)

Usual Hardware-Software Trend Comparison

time

Log N Number of transistors in service

New Source Lines of Code/year

SW

HW

209/27/99

10 4

10 5

10 6

10 7

10 8

10 9

10 10

10 11

10 12

10 13

10 14

10 15

1950 1960 1970 1980 1990 2000

LOCS

0.00001

0.0001

0.001

0.01

0.1

1

10

100

1000

10000

100000

1000000

$

LOCS

Total $/LOCS

Total LOCS

Years

Lines of Code in Service: U.S. Dept. of Defense

219/27/99

ExpansionFactor

The ratioof machine

lines ofcode to a

source lineof code

1

10

100

1000

1960 1965 1970 1975 1980 1985 1990 20001995

Order of Magnitude Increase Every Twenty Years

MachineInstructions

MacroAssembler

High LevelLanguage

DatabaseManager

On-line

RegressionTesting

Prototyping

4GL

SubsecondTime

Sharing

SmallScaleReuse

Object Oriented

Programming

Large ScaleReuse

142113

8175

4737.5

30

15

3

475638

Projection

Trends in Software Expansion (Bernstein, 1997)

229/27/99

• Major advances in effective software productivity

• Major advances in ability to build large software systems

– But with only partial intellectual control

• Advances are due to several sources– IT components research – Software engineering research– Industry process improvement– All are critical to success

Software Trends to Date

239/27/99

Critical Success Factors for SE Research Programs

• Emphasize scientific foundations– Clear hypotheses; careful measurements; repeatability – Evaluated w.r.t. alternatives and domain of applicability

• Broaden empirical understanding of software phenomenology– Enables focus on high-leverage problems and solutions

• Skate to where the puck is going (Gretzky)– Anticipate and address future problems

• Maintain a balanced research portfolio– Evolution/revolution; basic/applied; theory/systems

• Expand horizons via Grand Challenges• Stimulate “out of the box” ideas

– New metaphors: biology, sociology, economics

• Stimulate university-industry collaboration; transition into practice

249/27/99

Investment Strategy Guidelines:SE Research

• Comparable level of investment in empirical studies of software phenomena

• Increasing levels of investment in experimental research– Initially medium-scale trials– Where feasible, followed by industry

co-funded larger trials

259/27/99

Workshop Conclusions• Software Engineering technology advances have been

critical to IT advances to date– Along with software application technologies

• Software Engineering technology advances are critical to future IT performance– Along with software application technologies

• Need a balanced portfolio of SE research– Energized by applications-oriented Grand Challenges

• Research strategy should include other critical success factors– Scientific foundations; empirical root-cause studies;

anticipate future problems; “out of the box” ideas; university-industry collaboration; transition into practice

269/27/99

Software Engineering Technology Transition Challenges

• Adoption requires behavioral change• Marketplace often makes “fixing it later” more attractive than

“doing it right the first time”– “The IT industry expends the bulk of its resources, both

financial and human, on rapidly bringing products to market.” -PITAC Report, p.8

• Strong coupling among technologies, processes, acquisition practices, cultures

• Rapidly evolving commercial technology• Slowly evolving Government acquisition practices• Risk-averse program managers• Leaky intellectual property• Payoffs take a long time to demonstrate

– And are hard to trace back to particular technology insertions

279/27/99

Expected Payoff: Practical Achievement of No-Surprise Software • Expanding domain of applications with

well-understood solution approaches– Feasible to complete project with no surprises

(overruns, user mismatches, performance/reliability problems)

– Mix of cycle time, cost, functionality, levels of service matched to stakeholder needs

• Clear demarcation of no-surprise boundary– Identification of likely problems outside

boundary– No surprise that problems will be encountered

289/27/99

Prototype of No-Surprise Boundaries: Specification Methods

Formatted Specs

High

Requirements Stability

Low

Low Required Fault-freedom High

Informal

Formal Specs

•Exploratory Models

• Nuclear Release Software

• Security Kernels

• E-Commerce

299/27/99

SE Research Strategies: Workshop Approach

SE-Intensive Challenge Areas• Empowering people and groups• Weaving the new information fabric

Example challenge problems;SE research needs

SE research strategiesCritical success factorsfor SE research

Trends in IT demand,technology

PITAC challengesand IT goals