Galorath Incorporated 2004

AccuScope –A Tool For Relative Comparisons

Lee Fischman & Kenneth FongGalorath Incorporated

SCEA 2004Manhattan Beach, CA

2Galorath Incorporated 2004

Presentation OutlinePresentation Outline

AccuScope’s Role

Key Features

Four Step Sizing Process

Accuracy

Risk-driven estimates

Size Anything

Key Enabling Features• Shareable Repository of Known Items• Cut and Paste with Other Applications

3Galorath Incorporated 2004

AccuScope’s Role

Scope Software and Other Items

Earlier – provide reasonable size ranges prior to a detailed understanding of project  More Accurately – relative knowledge often results in more accurate up front estimates vs. other methods

More Efficiently – assist the sizing process with features making it easier and possible under more circumstances

4Galorath Incorporated 2004

Key Features

Mathematically Sound• Incorporates proven methods plus Galorath implementation• Utilizes pair-wise comparisons against a number of known and unknown items • Uses these comparisons to estimate unknown items

Repository of Known Items • Provides access to a repository of enterprise information on completed items • Can reside on your desktop or be deployed across an organization

Analogies • Provide “logical patterns” for sizing new items from typical functionality • Based on in-house experience, provides a common basis for characterizing scope

Smorgasbord of Quantitative Metrics• Source Lines of Code (SLOC)• Function Points• Function-Based Sizing• And many more customizable metrics…

5Galorath Incorporated 2004

Four Step Sizing Process

3. Perform various expert judgment comparisons

1. Identify Items to size

2. Choose references or analogies for comparison

4. View / Use results

6Galorath Incorporated 2004

Step 1: Identify Items To Size

Estimated items are those which will be sized.

7Galorath Incorporated 2004

Step 2: Choose References To Size

Reference items are those from which comparisons are made.

These items can be entered manually or obtained from a repository.

8Galorath Incorporated 2004

And (Or) Choose an Analogy

Analogies are optional.

Analogies are reference items based on past patterns.

Choose from various sets of analogies, each representing different sizing perspectives, or create your own manually.

9Galorath Incorporated 2004

Step 3: Ready For Comparisons

10Galorath Incorporated 2004

Step 3a: Compare Known & Unknown Items

Expert judgment establishes the relative size between ‘Tristate Net’ and ‘New Backup Site’.

11Galorath Incorporated 2004

Step 4: View / Use Results

AccuScope lets you view sizing information and estimates using a variety of reports and charts.

Reports are presented using different metrics depending on the size metric (source lines of code, function points, etc.)you are working with.

12Galorath Incorporated 2004

AccuScope’s AccuracyAccuScope’s Accuracy

For above test:Perfect information: no errant comparisonsEach bar above represents 250 randomized trials5 reference items compared to 3 estimated items7 stop slider range, 3 above ‘equal’ and 3 below

Accuracy of 'Likely' Outcome Under Different Scenarios

0.00%

1.00%

2.00%

3.00%

4.00%

5.00%

6.00%

7.00%

8.00%

9.00%

10.00%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Distributions: left-skewed, centered, right-skewed

MR

E

13Galorath Incorporated 2004

Factors Affecting AccuracyFactors Affecting Accuracy

Number of items being compared – more is better, since that places fewer constraints on information available to the algorithm.

Number of reference items – 1 can be enough, > 1 will improve results when comparisons are imperfect.

Appropriateness of reference items – a sensible range of items spanning the same range as estimated items is helpful although not required.

User’s assessments – the more accurate, the better. This is by far the most significant factor.

Granularity of comparisons – more granular is more accurate BUT this results in increasing ‘cognitive overload’ for the user.

Mapping between qualitative ratings and underlying ratios – the user’s conception of ‘Big’, ‘Bigger’, etc. should match the ratios they map to.

14Galorath Incorporated 2004

Least / Likely / Most RangesLeast / Likely / Most Ranges

Comparison Ratio Ranges

0.10

1.00

10.00

MuchBigger

Bigger SlightlyBigger

Equal SlightlySmaller

Smaller MuchSmaller

Qualifier Label

Com

paris

on R

atio

Ratios have a range – so that estimates capture the edge of intuition between (for example) ‘Slightly Bigger’ and ‘Bigger’.

User entries can have a range – so that the user can establish bounds on level of understanding.

Reference sizes can be entered as a range – in case the size count is imperfect.

15Galorath Incorporated 2004

Range of Inputs CreatesRange of Inputs CreatesEstimates Across A RangeEstimates Across A Range

Size ByComparison

Inputs* reference values* slider entries* comparison ratios

Lower

Likely

Highest

Lower

Likely

Highest

OutputsEstimated items

as a range

16Galorath Incorporated 2004

Configurable For ManyConfigurable For ManyEstimating ChallengesEstimating Challenges

Configured out of the box for software estimating, AccuScope may also be used to:

• rank risk• calculate hardware weight• determine colleagues’ ages• and so on…

17Galorath Incorporated 2004

Size Anything Using This Approach

Estimates based on past known items, analogies, manually created reference items, etc.Can size ANYTHING:• Hardware weight• Software size• Program office staff size

AccuScope can access a repository of reference items, analogies can be customizedEstimates are output as a range, for use in risk-based estimatingAnything can be sized by relative analysisPercentage or qualifier (“Smaller”, “Larger”, est.) ratings

18Galorath Incorporated 2004

Repository

Uses a variety of database types, including SQL Server or simpler, client-side solutions

Repository may be shared across the enterprise, so many users can access the same information

Online / offline modes

Easy editing of repository entries

19Galorath Incorporated 2004

Cut and Paste With Other ApplicationsCut and Paste With Other Applications

Copy to and fromMicrosoft Excel

and other applications

Copy / pasteintegration

with SEER-SEM

Microsoft ExcelAccuScope

SEER-SEM

20Galorath Incorporated 2004

Summary

AccuScope enables:

Reasonable size ranges prior to a detailed understanding of the project

Sharing of corporate knowledge

Customization to a wide variety of estimating challenges