Upload
emil-barnett
View
228
Download
0
Embed Size (px)
Citation preview
Lecture 17“Does every inspection need a meeting?”
FOR0383 Software Quality Assurance
04/21/23 1Dr Andy Brooks
Source: Votta, L G (1993) Does Every Inspection Need a Meeting? Proceedings of the first ACM symposium on the Foundations of Sofware Engineering, SIGSOFT’93, pp 107-114. ©ACM
n = 4
21.04.23 Dr Andy Brooks 2
The survey data
Reason Authors & Managers (29)
Synergy 79% (23)
Education 62% (18)
Schedule 48% (14)
Competition 34% (10)
Requirement 28% (8)
Qu: Why do you think inspection meetings are held?Responses coded from answers.All other reasons given occurred with a frequency of less than 20%.
Always provide data about what a % means.
21.04.23 Dr Andy Brooks 3
Top five reasons why meetings are held.
1. SynergyThe idea is that working together as team produces a better performance than the sum of the performance of individual team members. Synergy has been clearly demonstrated in many areas. Planting a tree as an individual is difficult, but not if you work in pairs.
However, synergy has not been clearly demonstrated for inspection meetings.
“If this effect occurred in the software inspection process, we would see many inspection faults found only by holding a meeting.”
samvirkni
21.04.23 Dr Andy Brooks 4
2. Education
The idea is that new employees learn from experienced reviewers how the software development process works, and what makes a good reviewer.
However, this might be “the blind leading the blind”. Sending employees on proper training courses about inspections might well be more efficient and effective.
Top five reasons why meetings are held.
21.04.23 Dr Andy Brooks 5
3. Schedule deadlines
People have a planned meeting to work towards.
Scheduling events such as inspection meetings helps with the time management of a project.
Top five reasons why meetings are held.
21.04.23 Dr Andy Brooks 6
4. Competition
Employees make their contribution to defect finding in public and so can earn the respect of their peer group. Employees try to improve their inspection performance.
However, managers can easily destroy teamwork by rewarding individual performance.
“Why should I devote a lot of time to inspecting, if it is always Joe who gets the salary bonus?”.
Top five reasons why meetings are held.
21.04.23 Dr Andy Brooks 7
5. Requirement
Holding a meeting to collect individual reviewer defects is required by the inspection process.
Having a well-defined process helps with the management of a project.
Top five reasons why meetings are held.
Figure 1. Typical Time Line for Inspection Process ©ACM
21.04.23 Dr Andy Brooks 8
“Typical values in a large software development for a feature of 600 NCSL are 10, 10, and 10 working days for Tinspection, Tcollection, and Trepair, respectively.”
L G Votta
21.04.23 Dr Andy Brooks 9
The development interval
• The longer money is borrowed, the more difficult it is to realise a profit.
• You get less and less responsive to customer needs and marketing conditions as you take longer and longer to develop something.
• Waiting for the collection meeting (Tcollection) to happen can be very costly.– hazard cost of being late to market– carrying cost if the author is not working on the product– rework cost if the author continues working on the product only
to have this work invalidated by the results of the inspection meeting
TinspectionTcollectionTrepair TinspectionTcollectionTrepair TinspectionTcollectionTrepair..................
21.04.23 Dr Andy Brooks 10
When can n people have a meeting?• Suppose
– each day comprises 4 two-hour slots
– m is the average number of meetings per day
– p is the probability a two hour slot is occupied by a meeting (= m/4)
– q is the probability that a two hour slot is unoccupied (= 1-p)
– assume p is the same for each person and independent
• The probability of any slot being available for n people is:
• The average number of days that elapse before the meeting occurs is:
nm
41
14
14
1n
m
n = 5
21.04.23 Dr Andy Brooks 11
© ACM
Keep the number of meetings to a minimum and the number attending small.Exception: architectural design meetings for a large system.
Average number of meetings per day.
21.04.23 12
Is there enough synergy?
Each point represents the number of faults undetected in individual preparation divided by the total number recorded at an inspection meeting.
4% average
© ACM
Always provide information about what an error bar means.
actual measurements
21.04.23 Dr Andy Brooks 13
The severity of a defect is important.
• Even if the inspection meeting yields only another 4%, the meeting will be invaluable if you find a showstopper.– A showstopper is a defect that will cause a system crash in the field
(such as null pointer dereference).
• Holding inspection meetings is, however, unnecessary if only additional minor defects are found.– This is especially true if static analysis tools can detect all the minor
defects (violations of coding conventions, for example).
• Inspection data needs to be recorded and analysed to help make decisions about the value of holding an inspection meeting.
• Common sense can also be applied:– “...this part of the requirements, design or code is so important that
we should have an inspection meeting.”
21.04.23 Dr Andy Brooks 14
False positives and lost defects.
• During individual preparation, inspectors can soemtimes report issues that are not defects (false positives).– An inspection meeting usually eliminates most of the
false positives.
• During the inspection meeting, some issues found in individual preparation or at the meeting do not get recorded (lost defects).– Lost defects typically represent a small percentage of
the total. (L G Votta quotes a figure of 4%.)• ... the benefit from synergy was measured at 4%.
21.04.23 Dr Andy Brooks 15
What are the alternatives to holding an inspection meeting?
Deposition Meetings• The moderator and author meet with single reviewers. (3
person meetings.)• There is no reading activity: a deposition meeting collects
the defects found from the reviewer.• The moderator acts as the recorder.• The moderator and author have to attend more meetings.• There is no meeting schedule delay as depositions can
be taken in a day.
21.04.23 Dr Andy Brooks 16
What are the alternatives to holding an inspection meeting?
Direct deposition hand over• The moderator collects all the individual logs.• The moderator can simply hand over the logs to
the author.• Or, the moderator creates a master defect report
which is then handed over to the author.
• The author and reviewers never physically meet and can even be located in different countries.
• Reviewers can use e-mail to report defects.• Reviewers can use general or special purpose
groupware to report defects.
21.04.23 Dr Andy Brooks 17
What are the alternatives to holding an inspection meeting?
Correspondence
IBIS - Internet Based Inspection Systemhttp://sourceforge.net/projects/ibis/
Adobe Acrobat 9http://www.adobe.com/products/acrobat/
21.04.23 Dr Andy Brooks 18
Inspection data needs to be recorded and analysed to help make decisions about the value of holding an inspection meeting.
What is the synergy rate?Are showstoppers being found?What is the lost defect rate?How big is the meeting schedule delay?
Common sense can be applied: “...this part of the requirements, design or code is so important that we should have an inspection meeting.”
almenn skynsemi