Measuring Quality: A Cornerstone of Theory in Software Engineering

Measuring Quality: A Cornerstone of Theory inSoftware Engineering

Howell Jordan

Lero - the Irish Software Engineering Research CentreUniversity College Dublin

November 2012

Howell Jordan 1/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Introduction

Talk Outline

I Internal qualities of software are important

I There is no perfect design; evolution is inevitable

I Software is especially evolution-prone

I Case studies of product qualities in engineering history

I When product theories are incomplete: Tacoma Narrows 1940

I Building product theory: Langley Field experiments 1919-1941

I A useful, general theory of software engineering should becapable of product quality prediction

Howell Jordan 2/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Internal Quality

Three Reasons Why Software Evolution Matters

1. Customers often don’t know what they want

“I’m not sure, but I’ll know it when I see it” - Barry Boehm,paraphrasing a typical software customer

2. There is no such thing as a perfect design

“The problems of multiple users or changing fashion or newaesthetics will always be lying in wait” - Henry Petroski

3. The variation will be in the software

“Most car manufacturers now offer engines with differentcharacteristics ... frequently these engines ... differ only in thesoftware of the car engine controller.” - Svahnberg et al 2005

Howell Jordan 3/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Internal Quality

Defining, Measuring, and Predicting Evolvability

Too many models, not much evidence, no consensus. McCall 1977:

See also ISO 9126, Fenton and Pfleeger, Henderson-Sellers, ...

Howell Jordan 4/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Historical Case Studies

Source Materials

I Problem: researchers in other engineering disciplines rarelymention epistemology or research methodology!

Henry Petroski Walter G. Vincenti

Howell Jordan 5/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Historical Case Studies

Tacoma Narrows Bridge Collapse, 1940

I 2-lane road suspension bridge over the Puget Sound,Washington

I 2800-foot span, the 3rd longest in the world at the time

I Ambitiously economical design ...

I ... but comfortably within limits suggested by 1930s structuralengineering theory

I Well-documented collapse due to aeroelastic fluttering

Aeroelastic fluttering in aircraft wings was already mostlyunderstood by late 1930s!

Howell Jordan 6/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Historical Case Studies

Tacoma Narrows Bridge Collapse, 1940

I 2-lane road suspension bridge over the Puget Sound,Washington

I 2800-foot span, the 3rd longest in the world at the time

I Ambitiously economical design ...

I ... but comfortably within limits suggested by 1930s structuralengineering theory

I Well-documented collapse due to aeroelastic fluttering

Aeroelastic fluttering in aircraft wings was already mostlyunderstood by late 1930s!

Howell Jordan 6/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Historical Case Studies

Langley Field Aeronautical Laboratory, Virginia

Howell Jordan 7/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Historical Case Studies

Langley Field Aircraft Experiments, 1919-1941

I Given an aircraft design, what will it be like to fly?

I What is flyability, anyway?

I Lengthy series of flying qualities experiments

I Dozens of aircraft tested

I Ground experiments on pilots and controls

I Parallel program of instrument development

Surprising discovery: the most important attribute of flyability isstick force per g !

Howell Jordan 8/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Historical Case Studies

Langley Field Aircraft Experiments, 1919-1941

I Given an aircraft design, what will it be like to fly?

I What is flyability, anyway?

I Lengthy series of flying qualities experiments

I Dozens of aircraft tested

I Ground experiments on pilots and controls

I Parallel program of instrument development

Surprising discovery: the most important attribute of flyability isstick force per g !

Howell Jordan 8/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Conclusions

Implications for Software Engineering Theory

I A thorough understanding of product qualities is vital

I Neglecting them can be disastrous

I Gaining that understanding is difficult but possible

I Complex socio-technical phenomena can still be quantified

I Development of measures and measuring instruments is key

I Empirical research may be prerequisite to theory

Thank You

Howell Jordan 9/9

Measuring Quality: A Cornerstone of Theory in Software Engineering Conclusions

Implications for Software Engineering Theory

I A thorough understanding of product qualities is vital

I Neglecting them can be disastrous

I Gaining that understanding is difficult but possible

I Complex socio-technical phenomena can still be quantified

I Development of measures and measuring instruments is key

I Empirical research may be prerequisite to theory

Thank You

Howell Jordan 9/9