www.testimation.com
Sunday,November5,2017 1
ApplicatorSuiteUserInstructions |v1.0forUseCase/UserStory |TEstimator1 |TEstimator2 |Testimator3 |REstimator |@RiskAuthor/s |TestimationTeamEngineeringtoolsforthescienceofestimation&riskmanagement
www.testimation.com
Sunday,November5,2017 2
ApplicatorSuite......................................................................................................................................11. Introduction................................................................................................................................42. TheBigPicture...........................................................................................................................53. TheApplicatorProductSuite................................................................................................6A. Composition&Architecture.............................................................................................................6B. FeatureList...............................................................................................................................................81. TEstimator1...........................................................................................................................................................82. TEstimator2...........................................................................................................................................................93. TEstimator3........................................................................................................................................................104. REstimator..........................................................................................................................................................105. @Risk....................................................................................................................................................................116. AEstimator..........................................................................................................................................................117. PEstimator..........................................................................................................................................................12
4. HowtoexecuteaTEstimator1Estimate..........................................................................15A. Applicability...........................................................................................................................................15B. EstimationProcess..............................................................................................................................15C. HowtoexecuteaQuEstimateorLEstimate.............................................................................151. Inputs....................................................................................................................................................................152. Configuration.....................................................................................................................................................15
D. HowtoexecuteanEstimateinBasicView...............................................................................16E. HowtoexecuteanEstimateinAdvancedView......................................................................16
5. HowtoexecuteaTEstimator2Estimate..........................................................................17A. Applicability...........................................................................................................................................17B. EstimationProcess..............................................................................................................................17C. Procedure................................................................................................................................................171. Inputs....................................................................................................................................................................172. Configuration.....................................................................................................................................................183. Analysis|Historical&ProjectedSolutions..........................................................................................194. QualityOverride...............................................................................................................................................205. RiskOverride.....................................................................................................................................................216. GeneralConsiderations.................................................................................................................................23
6. HowtoexecuteaTEstimator3Estimate..........................................................................24A. Applicability...........................................................................................................................................24B. EstimationProcess..............................................................................................................................25C. TEstimator3UserScenarios.............................................................................................................25
7. Definitions................................................................................................................................27A. CalibrationFactor................................................................................................................................27B. Configuration.........................................................................................................................................27C. ConversionRatio..................................................................................................................................27D. CriticalValue..........................................................................................................................................28E. Defect-FreeConfidence.....................................................................................................................28F. DIT’sperTestCase..............................................................................................................................28G. DynamicInformationTests(DIT’s).............................................................................................28H. ExecutedTestCases...........................................................................................................................28I. Framework...............................................................................................................................................28J. FrameworkProbabilities...................................................................................................................28K. FunctionalProcess..............................................................................................................................29L. ProgressionTesting............................................................................................................................29M. QualityAssuranceArchitecture....................................................................................................29N. QualityAssuranceStrategy.............................................................................................................29O. QualityOverrideSimilarity.............................................................................................................29P. RegressionTesting..............................................................................................................................30Q. RequiredTestCases...........................................................................................................................30R. RiskExposure........................................................................................................................................30S. RiskMitigation.......................................................................................................................................30
www.testimation.com
Sunday,November5,2017 3
T. RiskOverrideSimilarity....................................................................................................................30U. RiskQuotient.........................................................................................................................................30V. Scope.........................................................................................................................................................31W. SystemUnaffected..............................................................................................................................31X. UseCase....................................................................................................................................................31Y. UserStory................................................................................................................................................31Z. TestCase..................................................................................................................................................32AA. TestCase&DefectComplexity...................................................................................................32BB. TestScenario.......................................................................................................................................32
8. ApplicatorFunctionalProcesses.......................................................................................33A. FunctionalProcessList.....................................................................................................................33B. UseCase(UserStory)List................................................................................................................34C. HowtoCalculate(orestimate)theConversionRatio..........................................................34D. UserWorkFlows(FunctionalProcessMap)..............................................................................35
www.testimation.com
Sunday,November5,2017 4
1. Introduction
****ImportantNote****ThepurposeofthisdocumentistoactasacompaniontotheApplicatorSuite.Pleaserefertoitasrequired.Forbrevity&practicality,mostimages&screenshotswithintheApplicatorSuiteofToolshavebeenomittedfromthisdocumentduetotheintendedcompanionrelationship.****EndNote****
Have you ever wondered why your computer crashes so frequently but your car alwaysstarts?Theanswerissimple.Humanshavebeenmakingthingsformanythousandsofyears,butthe commercial software industry is only decades old. We have leveraged off this hardwaremanufacturing experience to develop a suite of products that are predictive & true riskmanagement tools. At Testimation, we believe that risk management should rarely be aboutqualitative high-level statements, rather, it should be about hard quantitative numbersexpressed as a percentage value. For example; “it's too risky” is a verydifferent statement to“there'sa23%probabilityofsuccess”.Theformeriscommonpracticeinriskmanagement,thelatterisTestimation.
Wearepassionateaboutsoftwarequality&engineeredsolutions.WebelievethatforfartoolongintheInformationTechnology(IT)sector,emphasishasbeenondevelopmentcostsratherthan product quality. This emphasis has arisen, not because organizations are mean withmoney, but because a scientific approach to inform business sponsors of risk has not beenmainstreaminIT.However,intheengineeringsector,thespecificationsoffailureprobabilitiesare commonplace. Stories appear too often in themedia about catastrophic software failuresinconveniencingorendangeringcustomers.Webelievethattheprimarycauseofthisisduetobusinesssponsorssimplynotbeingawareoftheriskoffailurebecauseithas(probably)neverbeenpresentedtothemasapercentagevalue.Typically,only“words”areusedtodescribetheriskofsoftwarefailure.Theproblemwiththisapproachisthatitishighlysubjective&dependsas much upon the person receiving the information, as it does upon the person sending it.Imagineifapharmaceuticalcompanyreleasedexperimentalcuresbasedupontheconceptof“itshouldbe fine”, rather thanstatisticallymeaningful clinical trials;howreadywouldyoube togivetheirdrugstoyourchildren?At Testimation, we want to change things ......
When risk is presented as a numerical probability of failure, human reaction to thisinformation is farmore sober thanwhen it is presented in qualitative forms only. If your ITprovidersorprocurementadvisersaregivingyou“words”¬numbers,thenyoushouldturntoTestimation.“Words”areeasytoproduce,theyrequirelittleeffort&onedoesnotneedtobean expert to “talk” (e.g. sales people); but only experts will give you hard numbers from theapplication of engineering methodologies. If your software development estimates are beingdelivered to you without the number of tests being specified, or without the probability offindingDefectsbeingexpressedwithhardnumbers,thenyouareplayingRussianroulette&youmay need to re-define success throughout the Project Life-Cycle to make it appear tostakeholdersthatyouhavedeliveredsuccessfully;thisoccursfarmorecommonlythanyoumayrealise.However,withourtools&services,youcandesigntheQualityAssurancestrategythatsuits your budget & time constraints, with confidence that you're aware of the risk & thepotentialimpactuponyourcustomers,users&business.
The Testimation Team brings formalized & verified engineering techniques to thedisciplinesofestimation&riskmanagement;specificallytothesoftwaredevelopmentindustry.It utilizes the same Quality Assurance principles as the manufacturing sector & highperformance industries such as aerospace, commercial aviation & medical research.Pharmaceutical companies do not release products to themarket unless the potential failurerate is numerically understood& the riskmitigated. Sowhy is society soprepared to releasesoftwaretouserswhenthedefect-freeconfidenceisonlydescribedwithwords(ifatall)¬hard numbers? ….We have addressed this question by developing engineering tools for the
www.testimation.com
Sunday,November5,2017 5
science of estimation & risk management. Our suite of tools empowers clients & serviceproviders to present scientific arguments supporting their Quality Assurance & riskmanagementprofiles.Otherstalkofriskinsubjective&salesmanshiplanguage;wequantifyitwithhardscience&hardnumbers.Ourproducts&servicesofferfourkeyfeatures:
1. Theworldsfirstsuiteofscientificallyformulated,web-deliveredestimationtools
2. ThenumberoftestsrequiredtobeexecutedforthelevelofQualityAssurancespecifiedbyprojectgovernance
3. Theprobabilityoffindingdefectsinaccordancewithprojectgovernancespecifications
4. Development,testing&projectmanagementeffort
2. TheBigPicture
The Testimation Team has decades of experience in Hardware + Software QualityAssurance, Process Re-Engineering & Risk Management across multiple industries, such asInformation Technology, Business Technology, Manufacturing, PetroChem, Utilities, Food,Government,Financial, Insurance,Construction,Aluminium&Retail.Fromourexperience,wehave learned that some industries possess significantly greater levels of Quality Assurance&Risk Management controls than others. For example, in 1969, the United States of AmericaplacedmenupononthemoonwithlesscomputingpowerontheApollo11commandmodulethanexistsonyoursmartphonetoday.Thishistoricalfactspeaksvolumesabout“whatcanbe”& “what is” within the IT sector. The harsh reality of 21st century IT is that, although ourtechnologyhasdramaticallyadvancedoverrecenthistory,good&healthyengineeringpracticeshave (largely) not been introduced or totally ignored, sector wide, across IT. This fact issurprisingly easy to prove; for example, howmany software development projects have youseen thathaveever referenced therelevant international standards in relation to theproductbeingbuilt,orthedevelopmentprocessbeingundertaken?Ifyou’rehonestwithyourself,you’llprobablyanswer“never”,oratleast,“veryrarely”.Manyinternationalsoftwarestandardsexist&manymorearebeingdevelopedasyoureadthisdocument.
Whilst it is exceedingly common practice in themainstream engineering sector to designsolutionsaccordingtogeneral&specificstandards,withintheITsectorthisdoesnotseemtooccur with any significant frequency. The primary reason for this may be because softwaredevelopersareoftennot formally trainedorqualifiedengineers& therefore, lacksomeof thefundamental design skills acquired during the university indoctrination process. One is notrequiredtobeuniversityeducated&examinedtobetitled-“adeveloper”,butoneisrequiredtopossesstheseattributesinordertobetitled“anengineer”(globally).Infactasageneralization,in many places around the world, the only professions whose signatures constitute a legaldocument are doctor, lawyer & engineer; all of which require formalized & recognizeduniversity training. So why is this important? It is important because repeatable processesrequire standardization, & standardization comes from national & international bodies ofacceptance such as the International Organization for Standards (ISO). For example, imaginethatyouhadtostartyourcar inadifferentmannereachtimeyou intendedtouse it. Imaginethatonetimeyouusethekey,thenexttimeyoutaponarearwheel,&afterthatyoumustdosomethingdifferenteachtimeyouwishtostartthecar.Thelessonhereisthateffectivedesign,development,delivery&UserAcceptancemodelsrequirestandardization,&standardizationisa hallmark characteristic of a formal engineering design process. A process involves theexecution of actions & one of the very first required actions is estimation. The SoftwareDevelopmentLife-Cycleisnoexception;asignificantportionofitinvolves;TheSoftwareTestingLife-Cycle....Itallbeginswithanestimate
1. Whyisestimationimportanttoaclient?• Becausethisiswhenaclientsetsthebudget
www.testimation.com
Sunday,November5,2017 6
2. Whyisestimationimportanttoaserviceprovider?• Becausethisiswhenprofitorlossisdefined
3. WhencanourProductSuitebeused?
• Insituationswherethequalitycapabilityshouldbeknown1. Bids/Tenders,Projects,Audits,Operations,UAT2. AssessmentsofServiceProviders&VendorSoftware
4. WhatdoesusingourProductSuitemeanforyou?
Users Uses1. BoardofDirectors2. CEO's,CIO’s,CTO’s,CPO’s3. TM's,DM's,PM's,RM's4. BusinessSponsors5. Procurement6. SystemIntegrators7. SecurityAgencies8. SoftwareHouses9. Corporations10. Government11. Military
1. Savemoneybyforecastingaccurately&efficiently2. Define,manage&mitigateriskproperly3. DesignQualityAssurancestrategiesfittingyourbudget4. Conformity,repeatability&transparencyofprocess5. Thesubjectivityplaguingestimationhaslargelyevaporated6. EstimatesareEngineeringbased7. FacilitatesContinuousImprovementProcesses8. Respondrapidlytochangesinscope9. ScalefromanHistoricalSolutiontoaProjectedSolution10. CalculatetheprobabilityoffindingDefects11. Calculatetherequirednumberoftests
3. TheApplicatorProductSuite
A. Composition&Architecture
The Applicator Product Suite incorporates five (5) key Use Cases (User Stories). Each ofthese five (5) User interactions executes a distinctly different process. The Product Suite iscomposedofthefollowing:
1. TEstimator1|estimatestheminimum&maximumnumberoftestsi
2. TEstimator2 | estimates the exact number of tests when scaling from an HistoricalSolutiontoaProjectedSolution
3. TEstimator3 | estimates the exact number of tests when scaling from an Historical
SolutiontoaProjectedSolution
4. REstimator(StandAlone)|estimatestheminimumnumberofRegressiontests5. @Risk(StandAlone)|UserAcceptanceTesting(UAT)RiskVisualisation
Additionalembeddedcapabilitiesare:
1. AEstimator|comparestwoestimates(sidebyside)
2. PEstimator|estimatesthetesteffortinMan-Days
ii.e.Thelower&upperlimits
www.testimation.com
Sunday,November5,2017 7
KeydifferencesbetweenTEstimator1,2,3are:
1. TEstimator1requiresone(1)input• Scopeonly
2. TEstimator2,3requirethree(3)inputs
1. HistoricalScope2. HistoricalTestCases3. ProjectedScope
KeydifferencesbetweenTEstimator2,3are:
1. TEstimator2 can scientifically assess the level of Quality Assurance of Historical
Solutions,butislessflexibleiithanTEstimator3
2. TEstimator3 provides greater flexibility than TEstimator2, but cannot scientificallyassessthelevelofQualityAssuranceofHistoricalSolutionsiii
Applicator Product Suite architecture & solution intelligence may be represented by thefollowingVenndiagram;
iii.e.TheQualityOverridefunctionisoptionallyactivatediiii.e.TheQualityOverridefunctionismandatorilyactivated
www.testimation.com
Sunday,November5,2017 8
B. FeatureList
WhatisaFeature&Why?
AFeature isany InputorOutput.TheyareFeaturebecause theydonotoccurnaturally;a
person is required to physically design, build & implement them. Even if the Features arerepeated,ittakeshumanwill&efforttorepeatthemondifferentscreensorreports.Everything,withoutexception,inallApplications,requireLines-of-Codetobecreated,copiedormodifiedtoperform the functions we desire. The choice of Input & the form of Output requires humandecision-makingintermsofneed&suitability.Forthesereasons,Inputs&OutputsareFeaturesbecausetheyrequirethemindofahumanbeingforthemtobecometangible.
1. TEstimator1
A. Userinterfaces1. QuickEstimateView2. LeanEstimateView3. BasicView4. AdvancedView
B. UserInputMetric5. FunctionalProcesses6. UseCases7. UserStories8. TestScenarios
C. Userscanspecifyprioritization&riskaspartoftheirtestingstrategy9. IntermsofCalibrationFactoriv10. IntermsofRiskQuotientv
D. ProgressionTestingTheminimumnumberofrecommendedProgressionTestCases11. IntermsofDynamicInformationTests12. IntermsofTestCasesThemaximumnumberofrecommendedProgressionTestCases13. IntermsofDynamicInformationTests14. IntermsofTestCases
E. RegressionTesting15. TheoptimalnumberofmanualRegressionTestCases16. TheoptimalnumberofautomatedRegressionTestCases
F. RiskManagement17. RiskExposurevi{β}18. RiskMitigationvii{α}19. TestCoverageofDevelopment(%)20. SystemUnaffectedbyDevelopment(%)21. TestCasecomplexity(%)22. Defectcomplexity(%)
G. General23. GraphicallydisplaysQualityAssurancearchitecture&strategy{α,β}24. RandomconfigurationgeneratorforUsertrainingpurposes
ivi.e.TestCasepriorityvi.e.ThelevelofAcceptableRiskvi%System-wideimpactofDevelopment{β}vii%ProbabilityoffindingDefects{α}
www.testimation.com
Sunday,November5,2017 9
2. TEstimator2
H. UserInputMetric25. FunctionalProcesses+TestCases26. UseCases+TestCases27. UserStories+TestCases28. TestScenarios+TestCases
I. TestCaseCeilingGuide HistoricalSolutions
29. 99%CoverageConfidence30. 99.6%CoverageConfidence31. 99.84%CoverageConfidence32. 99.93%CoverageConfidence33. 99.97%CoverageConfidence
ProjectedSolutions34. 99%CoverageConfidence35. 99.6%CoverageConfidence36. 99.84%CoverageConfidence37. 99.93%CoverageConfidence38. 99.97%CoverageConfidence
J. ProgressionTesting39. TheminimumnumberofrecommendedProgressionTestCases40. ThemaximumnumberofrecommendedProgressionTestCases
K. RiskManagement HistoricalSolutions
41. RiskExposureviii{δ}42. RiskMitigationix{γ}43. TestCoverageofDevelopment(%)44. SystemUnaffectedbyDevelopment(%)45. TestCasecomplexity(%)46. Defectcomplexity(%)47. TestApproachundertakenintermsofCalibrationFactorx48. TestApproachundertakenintermsofQuotientxi49. AssessmentofthelevelofQualityAssuranceoftheHistoricalSolution
ProjectedSolutions50. RiskExposurexii{ζ}51. RiskMitigationxiii{ε}52. TestCoverageofDevelopment(%)53. SystemUnaffectedbyDevelopment(%)54. TestCasecomplexity(%)55. Defectcomplexity(%)56. TestApproachtobeundertakenintermsofCalibrationFactorxiv57. TestApproachtobeundertakenintermsofQuotientxv58. AssessmentofthelevelofQualityAssuranceofHistoricalSolutions
viii%System-wideimpactofDevelopment{δ}ix%ConfidencethatallDefectswerefound{γ}xi.e.TestCasepriorityxii.e.thelevelofAcceptableRiskxii%System-wideimpactofDevelopment{ζ}xiii%ConfidencethatallDefectswillbefound{ε}xivi.e.TestCasepriorityxvi.e.ThelevelofAcceptableRisk
www.testimation.com
Sunday,November5,2017 10
L. General59. Users canspecifyQualityOverrideaspartof their interpretationof the testing
strategyintheHistoricalSolutionxvi60. UserscanspecifyRiskOverrideaspartoftheirtestingstrategyfortheProjected
Solutionxvii61. GraphicallydisplaystheQualityAssurancearchitecture&strategy,ofProjected
Solutions&HistoricalSolutionssidebyside{δ,γ,ζ,ε}62. Intelligence preventing unrealistic or impossible User-Defined Historical
SolutionsinfluencingProjectedSolutions(viaWorkFlowpreventionlogic)xviii63. RandomconfigurationgeneratorforUsertrainingpurposes
3. TEstimator3
M. UserInputMetric64. ThirteenvariantsofUserinputmetricavailablexix65. User-DefinedNotesfield(HistoricalSolution)66. User-DefinedNotesfield(ProjectedSolution)
N. ProgressionTesting67. The number of recommended Progression tests for new Projected Solutions
baseduponHistoricalSolutionsO. RiskManagement
HistoricalSolution68. Defect-FreeConfidence:see{η}
ProjectedSolution69. Defect-FreeConfidence:see{θ}
P. General70. GraphicallydisplaystheQualityAssurancearchitecture&strategy,ofProjected
Solutions&HistoricalSolutions(overlaid):see{η,θ}71. Users canspecifyQualityOverrideaspartof their interpretationof the testing
strategyintheHistoricalSolutionxx72. UserscanspecifyRiskOverrideaspartoftheirtestingstrategyfortheProjected
Solutionxxi73. Intelligence preventing unrealistic or impossible User-Defined Historical
SolutionsinfluencingProjectedSolutionsxxii74. RandomconfigurationgeneratorforUsertrainingpurposes
4. REstimator
Q. UserInputMetric75. TestCaseRepository(TCR)Size(i.e.theoverallnumberofTestCases)76. %EstimatedimpactupontheTestCaseRepositorybytheProject77. User-DefinedSamplingConfidence%78. AveragenumberofDynamicInformationTests(DIT’s)perTestCase
xvii.e.thelevelofExecutedRiskxviii.e.ThelevelofAcceptableRiskxviiiUsersarepreventedfrommovingtotheAEstimatorscreenxixe.g.Requirements,linesofcode,securitythreats,businessscenariosetc.xxi.e.thelevelofExecutedRiskxxii.e.ThelevelofAcceptableRiskxxiii.e.viaNormalizationoftheestimateatthePEstimatorscreen
www.testimation.com
Sunday,November5,2017 11
R. RiskManagement SamplingDistribution
79. Number of Functional Processes associated with the User-Defined SamplingDistributionConfidence
80. Number of Test Cases associatedwith the User-Defined Sampling DistributionConfidence
81. Risk Mitigation: i.e. system-wide coverage of the User-Defined SamplingDistributionxxiii{ν}
S. General82. Number of Functional Processes associated with the existing Test Case
Repository83. NumberofFunctionalProcessesimpactedbytheProjectchanges84. NumberofexistingTCRtestsimpactedbyProjectchanges85. Number of TCR tests associated with the User-Defined Sampling Distribution
Confidence86. System-widecoverageofexistingTCR:see{κ}87. System-widecoverageofexistingTCRimpactedbyProjectchanges:see{λ}88. Graphically displays Quality Assurance& Risk profiles of theTCR & the User-
DefinedSamplingDistributionsub-set{ν,κ,λ}89. RandomconfigurationgeneratorforUsertrainingpurposes
5. @Risk
T. UserInputMetric90. FunctionalProcesses91. UseCases92. UserStories93. TestScenarios94. UserAcceptanceTests(i.e.TestCases)
U. RiskManagement95. Defectcomplexity(%)96. %ProbabilityofUserAcceptanceTesting(UAT)findingDefectsxxiv{ρ}97. %FunctionalProcessesimpactedbyProjectchangesxxv{β}98. %FunctionalProcessesimpactedbyProjectchangesxiii&coveredbyUATxxvi
V. General99. Graphicaldisplay{α, β}100. RandomconfigurationgeneratorforUsertrainingpurposes
6. AEstimator
W. UserInputMetric101. TestCases
X. RiskManagement102. TEstimator1RiskQuotient103. TEstimator2RiskQuotient104. AlternativeEstimateRiskQuotient105. TEstimator1CalibrationFactor106. TEstimator2CalibrationFactor107. AlternativeEstimateCalibrationFactor
xxiii%ProbabilityoffindingDefects{ν}xxivi.e.System-wide{ρ}xxv%System-wideimpactofDevelopment{β}xxviTestCoverageofDevelopment
www.testimation.com
Sunday,November5,2017 12
108. TEstimator1DIT’sperTestCase109. TEstimator2DIT’sperTestCase110. AlternativeEstimateDIT’sperTestCase111. %TEstimator1RiskExposure:see{β}112. %AlternativeEstimateRiskExposure:see{β}113. %TEstimator1RiskMitigationxxvii{α}114. %AlternativeEstimateRiskMitigationxxviii{ι}115. %TEstimator2RiskExposure{ζ}116. %AlternativeEstimateRiskExposure{ζ}117. %TEstimator2RiskMitigationxxix{ε}118. AlternativeEstimateRiskMitigationxxx{ι}119. TestCoverageofDevelopment(%)byTEstimator1120. TestCoverageofDevelopment(%)byAlternativeEstimate(TEstimator1screen)121. TestCoverageofDevelopment(%)byTEstimator2122. TestCoverageofDevelopment(%)byAlternativeEstimate(TEstimator2screen)123. SidebysidecomparisonbetweenTEstimator1&theAlternativeEstimate124. SidebysidecomparisonbetweenTEstimator2&theAlternativeEstimate125. %SimilaritybetweentheUserdefinedestimate&thealternativesourcebyArea126. %SimilaritybetweentheUserdefinedestimate&thealternativesourcebyTest
CasepopulationY. General
127. GraphicallydisplaysQualityAssurancebetweentheUserdefinedestimate&thealternativesource
128. RandomconfigurationgeneratorforUsertrainingpurposes
7. PEstimatorZ. UserInputMetric
129. ProgressiveTestCaseDesignThroughput130. ProgressiveTestCaseModificationThroughput131. ProgressiveTestCaseExecutionThroughput132. RegressiveTestCaseExecutionThroughput133. QualityAssurancePersonnelResourcing134. %ManualProgressiveTesting135. %AutomatedREstimatorTestCases136. %AnticipatedTestCaseFailures
AA. RiskManagement TEstimator1
137. FunctionalProcesses138. TestCases139. RiskExposurexxxi{β}140. RiskMitigationxxxii{α}141. RiskMitigationxxxiii{ν}
TEstimator2142. FunctionalProcesses143. TestCases
xxvii%ProbabilityoffindingDefects{α}xxviii%ProbabilityoffindingDefectsrelatingtoTEstimator1{ι}xxix%ProbabilityoffindingDefects{ε}xxx%ProbabilityoffindingDefectsrelatingtoTEstimator2{ι}xxxi%System-wideimpactofDevelopment{β}xxxii%ProbabilityoffindingDefectsviaProgressionTesting{α}xxxiii%ProbabilityoffindingDefectsviaRegressionTesting{ν}
www.testimation.com
Sunday,November5,2017 13
144. RiskExposurexxxiv{ζ}145. RiskMitigationxxxv{ε}146. RiskMitigationxxxvi{ν}
TEstimator3147. FunctionalProcesses148. TestCases149. RiskExposurexxxvii{ξ}150. RiskMitigationxxxviii{ο}151. RiskMitigationxxxix{ν}
REstimator152. FunctionalProcesses153. TestCases
BB. QualityAssuranceBreakdown TEstimator1
154. ProgressiveTEstimatorTestCasestobeDesigned155. RegressiveTEstimatorTestCasestobeModified156. TEstimatorTestCasestobeExecuted157. REstimatorTestCasestobeExecuted158. AnticipatedDefects159. TotalExecutableTestCases
TEstimator2160. ProgressiveTEstimatorTestCasestobeDesigned161. RegressiveTEstimatorTestCasestobeModified162. TEstimatorTestCasestobeExecuted163. REstimatorTestCasestobeExecuted164. AnticipatedDefects165. TotalExecutableTestCases
TEstimator3166. ProgressiveTEstimatorTestCasestobeDesigned167. RegressiveTEstimatorTestCasestobeModified168. TEstimatorTestCasestobeExecuted169. REstimatorTestCasestobeExecuted170. AnticipatedDefects171. TotalExecutableTestCases
CC. ProjectManagement TEstimator1
172. Man-Days173. Man-Weeks174. Man-Months
TEstimator2175. Man-Days176. Man-Weeks177. Man-Months
TEstimator3178. Man-Days179. Man-Weeks180. Man-Months
xxxiv%System-wideimpactofDevelopment{ζ}xxxv%ConfidencethatallDefectswillbefoundviaProgressionTesting{ε}xxxvi%ProbabilityoffindingDefectsviaRegressionTesting{ν}xxxvii%System-wideimpactofDevelopment{ξ}xxxviii%ConfidencethatallDefectswillbefoundviaProgressionTesting{ο}xxxix%ProbabilityoffindingDefectsviaRegressionTesting{ν}
www.testimation.com
Sunday,November5,2017 14
DD. General181. GraphicallydisplaysprojectQualityAssurance&Riskprofiles{α,β,ν}182. GraphicallydisplaysprojectQualityAssurance&Riskprofiles{ζ,ε,ν}183. GraphicallydisplaysprojectQualityAssurance&Riskprofiles{ξ,ο,ν}184. RandomconfigurationgeneratorforUsertrainingpurposes
EE. TotalFeatureCount1. NumberofTEstimator1Reportfeatures =242. NumberofTEstimator2Reportfeatures =393. NumberofTEstimator3Reportfeatures =114. NumberofREstimatorReportfeatures =155. Numberof@RiskReportfeatures =116. NumberofAEstimatorReportfeatures =287. NumberofPEstimatorReportfeatures =568. TotalNumberofReportfeatures =1849. NumberofScreenfeatures=TotalNumberofReportfeatures=18410. TotalNumberofApplicatorProductSuitefeatures =368+
www.testimation.com
Sunday,November5,2017 15
4. HowtoexecuteaTEstimator1Estimate
A. Applicability
Uses UserInterfaces Users1. Bids2. Tenders3. RiskAssessment
1. QuEstimateViewQuickEstimate
2. LEstimateViewLeanEstimate
3. BasicView4. AdvancedView
1. BoardofDirectors2. ChiefExecutiveOfficers3. ChiefTechnologyOfficers4. SecurityAgencies5. ServiceProviders6. SystemIntegrators7. ProjectManagers8. DevelopmentManagers9. QualityAssuranceManagers10. Corporations11. Government
B. EstimationProcess
C. HowtoexecuteaQuEstimateorLEstimate
1. Inputs
1. SetupScope1. SelectFunctionalProcesses,UseCases,UserStoriesorTestScenariosfromthe
dropdownmenu2. Specifythedesiredvaluebyusingtheslider
2. Configuration
1. SpecifyorConfirmConversionRatio1. ThisistheaveragenumberofFunctionalProcessesyouexpecttohaveperUse
Case,UserStoryorTestScenario2. Thedefaultvalue forUseCases&UserStories is5, thedefaultvalue forTest
Scenariosis13. IfyoudonotknowtheConversionRatio,usethedefaultvalue
2. SpecifyorConfirmDynamicInformationTests(DIT’s)perTestCase1. ThisistheaveragenumberofDynamicDataFieldsyouexpecttocheckperTest
Case2. IfyoudonotknowtheDIT’sperTestCase,usethedefaultvalue
SetupScope
• AllViews
SpecifyorCon�irmConversionRatio
• AllViews
SpecifyorCon�irmDIT's
• AllViews
SpecifyorCon�irmCalibrationFactor
• BasicView• AdvancedView
SpecifyorCon�irmRiskQuotient
• BasicView• AdvancedView
OutputEstimate
• AllViews
www.testimation.com
Sunday,November5,2017 16
D. HowtoexecuteanEstimateinBasicView
1. See“HowtoexecuteaQuEstimateorLEstimate”2. Select a Calibration Factor Set-Point. The Calibration Factor (CF) is part of the
System-WideTestApproach,not justthecodebeingDevelopedorModifiedfortheProject
1. CF=0.25TargetsCriticalPriorityTests2. CF=0.5TargetsCritical&HighPriorityTests3. CF=0.75TargetsCritical,High&ModeratePriorityTests4. CF=1TargetsCritical,High,Moderate&LowPriorityTests5. CF=1.25TargetsallTests+25%Redundancy6. CF=1.5TargetsallTests+50%Redundancy7. CF=1.75TargetsallTests+75%Redundancy8. CF=2TargetsallTests+100%Redundancy9. SeeFrameworkProbabilitiesInformationtoassistwithdecisions
3. SelectRiskQuotientSet-Point1. TheRiskQuotientallowsaUsertotargetanacceptablelevelofriskinorder
tominimiseexpenditure2. SeeFrameworkProbabilitiesInformationtoassistwithdecisionsxl
E. HowtoexecuteanEstimateinAdvancedView
1. See“HowtoexecuteaQuEstimateorLEstimate”2. See“HowtoexecuteanEstimateinBasicView”3. ClicktheCalibrationFactorExpandbutton(ifrequired)
1. TheUserhasaccesstotheCalibrationFactorApplyCriticalValuebutton2. ThisfeaturemaybeinvokedtocomputetheminimumnumberofTestCases
providing100(%)TestCoverageofDevelopment.3. This featureprovidesaDevelopedorModifiedcode-onlyspecificTestCase
sizingsolutionwhenα=β4. ClicktheRiskQuotientExpandbutton(ifrequired)
1. TheUserhasaccesstotheRiskQuotientdial2. TheUserhasaccesstotheRiskQuotientApplyCriticalValuebutton3. ThesefeaturesprovidetheUserwiththeabilitytoreduceTestCasesizingby
controllingrisk4. These features may be invoked to compute the minimum number of Test
Casesproviding100(%)TestCoverageofDevelopment5. ThesefeaturesprovideaDevelopedorModifiedcode-onlyspecificTestCase
sizingsolutionwhenα=β
xlParticularly:TheprobabilityoffindingatleastoneDefect
www.testimation.com
Sunday,November5,2017 17
5. HowtoexecuteaTEstimator2Estimate
A. Applicability
1. The Projected Solution is scaled from an Historical Solution via the Defect-FreeConfidence
2. The Defect-Free Confidence is calculated scientifically by default, or may beoverridden&assignedavaluebytheUser
Uses Users1. In-houseProjects2. Bids3. Tenders
1. BoardofDirectors2. ChiefExecutiveOfficers3. ChiefTechnologyOfficers4. SecurityAgencies5. ServiceProviders6. SystemIntegrators7. ProjectManagers8. DevelopmentManagers9. QualityAssuranceManagers10. Corporations11. Government12. Military
B. EstimationProcess
C. Procedure
1. Inputs
1. SetuptheHistoricalSolution1. Select Functional Processes, Use Cases, User Stories or Test Scenarios from
thedropdownmenu2. Specifythedesiredvaluebyusingtheslider3. InputthenumberofExecutedTestCases
2. SetuptheProjectedSolution1. Specifythedesiredvaluebyusingtheslider
SetupScope
• HistoricalSolution• ProjectedSolution
SpecifyorCon�irmConversionRatio
• HistoricalSolution
SpecifyorCon�irmDIT's
• HistoricalSolution
SpecifyorCon�irmRiskQuotient
• HistoricalSolution• QualityOverride
• ProjectedSolution• RiskOverride
OutputEstimate
• ProjectedSolution
www.testimation.com
Sunday,November5,2017 18
2. Configuration
1. SpecifyConversionRatio1. This is the average number of Functional Processes you expect to have per
UseCase,UserStoryorTestScenario2. ThedefaultvalueforUseCases&UserStoriesis5;thedefaultvalueforTest
Scenariosis13. IfyoudonotknowtheConversionRatio,usethedefaultvalue
2. SpecifyDynamicInformationTests(DIT’s)perTestCase1. This is theaveragenumberofDynamicDataFieldsyouexpect to checkper
TestCase2. IfyoudonotknowtheDIT’sperTestCase,usethedefaultvalue
3. ConfigureQualityOverride1. The default configuration at page load is a scientific assessment of the
Historical Solution. At this juncture, the User must decide to override thescientificassessment(ornot)
2. If the User decides to utilise the scientific assessment; no further action isrequired
3. IftheUserdecidestooverridethescientificassessment,theUsermustassignaDefect-FreeConfidence to theHistorical Solutionbyutilising theSet-PointSliderorRiskQuotientDialxli
4. IftheUserwishestoreturntothescientificassessment,itmaybeachievedbyclickingthe“ApplyCriticalValue”buttonunderneaththeRiskQuotientDial
4. ConfigureRiskOverride1. Byinspectionofthegraphs&generalProjectconsiderations,theUserdecides
theappropriateconfigurationfortheProjectedSolution2. IftheProjectisconstrainedbytimeorbudget,theUsermaywishtobalance
theappetiteforTestingEffort(TestCases)withanacceptablelevelofDefect-FreeConfidence
3. The default Risk Quotient value is 0% (yielding a Defect-Free Confidence of99%). If the User wishes to focus on the Projected Code Changes only &minimise the Test Effort, the “Apply Critical Value “ button underneath theRiskQuotientDialmaybeutilised;theeffectofthisactionisreflectedinthegraphs
4. IftheCriticalValuefortheRiskQuotientappearsnegative,thisindicatesthatthe default scientific assessment of the Historical Solution has beenoverriddenbytheUser&it is likelythat insufficientTestingwasperformed.In this case, the User should consider utilising an alternative HistoricalSolutionasaframeofreferencefortheProjectedSolution
xliExamples1. TheUser believes that theHistorical Solution yielded an 80.22%probability that no
Defects exist (system-wide); therefore, the User selects a Risk Quotient Set-Point of50%.Thisrelatestoan80.22%probabilitythatallsystem-wideCriticalPriorityTestswereexecuted&passedsuccessfully
2. The User believes that the scientific assessment of Defect-Free Confidence is toocautious & recalls that the Historical Solution was implemented without significantincident. The User believes that the system-wide Defect-Free Confidence isapproximately95%.Consequently,theUserselectsaRiskQuotientSet-Pointof23%
3. The User believes that the scientific assessment of Defect-Free Confidence is toocautious & recalls that the Historical Solution was implemented without significantincident. The User believes that the system-wide Defect-Free Confidence isapproximately99%.Consequently,theUserselectsaRiskQuotientSet-Pointof0%
www.testimation.com
Sunday,November5,2017 19
3. Analysis|Historical&ProjectedSolutions
1. TheGraphsgeneratedcommunicate1. RiskExposure
1. HistoricalSolution1. Theproportionofsystem-wide functionality thatwasDevelopedor
Modified2. ProjectedSolution
1. The proportion of system-wide functionality to be Developed orModified
2. RiskMitigation1. HistoricalSolution
1. TheprobabilitythattheapproachtoTestingfoundallDefects2. ProjectedSolution
1. The probability that the approach to Testing will find at least oneDefect
3. TestCoverageofDevelopment1. HistoricalSolution
1. The proportion of Development that was covered by the TestApproach
2. ProjectedSolution1. TheproportionofDevelopmenttobecoveredbytheTestApproach
4. TheTargetedDistributionfortheHistoricalSolutionxlii5. TheTargetingDistributionfortheProjectedSolutionxliii
2. TestExecutionApproachcommunicates1. HistoricalSolution
1. ViaCalibrationFactor1. The composition of Critical, High, Moderate & Low Priority Test
Cases,whichshouldhavebeenExecuted2. ViaQualityOverride
1. The composition of Critical, High, Moderate & Low Priority TestCasestheUserclaimswereExecuted
2. ProjectedSolution1. ViaCalibrationFactor
1. The composition of Critical, High, Moderate & Low Priority TestCaseswhichshouldbeExecuted
2. ViaQualityOverride1. The composition of Critical, High, Moderate & Low Priority Test
CasestheUserclaimswillbeExecutedasaScaledSolution
xliiDescription1. AvisualrepresentationofthecombinedeffectoftheCalibrationFactor&RiskQuotient2. Communicates the effect of the Calibration Factor& RiskQuotient upon the Targeted
DistributionofFunctionalProcessesthatwereDevelopedorModified3. CommunicatesthattheUserfocusedTestingaroundthemeannumberofDataFieldsper
DatabaseRecordorthemostcommonlyusedsystem-wideFunctionsxliiiDescription1. AvisualrepresentationofthecombinedeffectoftheCalibrationFactor&RiskQuotient2. Communicates the effect of the Calibration Factor&RiskQuotient upon the Expected
(Targeting)DistributionofFunctionalProcessestobeDevelopedorModified3. CommunicatesthattheUserexpectstofocusTestingaroundthemeannumberofData
FieldsperDatabaseRecordorthemostcommonlyusedsystem-wideFunctions
www.testimation.com
Sunday,November5,2017 20
4. QualityOverride
1. This section allows theUser to Override the default (scientific) Analysis of theHistorical Solution by specifying the perceived level of Risk Quotientretrospectively
2. After entering the appropriate information in the Scope section, theUsermustdecideifthedefaultAnalysisisindicativeoftheactualresultsassociatedwiththeHistoricalSolutionxliv
3. IftheRiskQuotientsliderisselectedortheUserappliesthedial,thenthedefaultAnalysis has been overridden by the User in favour of his / her own opinion.Consequently, the Projected Solution utilises theOverriddenUser-SpecificationtocalculatetheRequiredTestCasesfortheProjectedSolution
4. CriticalValuexlv1. Unlike TEstimator1, the “Apply Critical Value” button does not graphically
illustratethevalueofRiskQuotientpreciselycorrespondingto100(%)TestCoverage of Development. It is not permitted to do this because it isconstrainedbytheactualnumberofTestCasesexecutedhistorically
2. If the “Apply Critical Value” button cannot be clicked, or has been clicked,then the Projected Solution utilises the default Analysis of the HistoricalSolutionbaseduponStatisticalMechanics&Probabilities
5. Defect-FreeConfidencexlvi1. Definition:TheprobabilityoftheHistoricalSolutionbeingDefect-Freebased
uponUserchoices2. ByselectingaRiskQuotientpre-setbuttonon theslider,orusing theRisk
Quotient dial to specify a value anything other than the Critical Valueindicatedbelowthedial,theUserclaimsthattheDefect-FreeConfidenceoftheHistoricalSolutionwasdifferenttothescientificvaluepredictedxlvii.Forexample,byselecting0%RiskQuotient,theUserclaimsthatthenumberofTest Cases Executed for the Historical Solution yielded a Defect-FreeConfidenceof99%.Similarly, if theUser selects50%RiskQuotienton theslider, the User claims that the Historical Solution had a Defect-FreeConfidenceof80.22%
xlivDescription1. Atpage load,orwhentheUserclicksthe“ApplyCriticalValue”button, theapplication
invokes the default Analysis; i.e. the scientific assessment of the Historical SolutionbaseduponStatisticalMechanics&Probabilities
2. The default Analysis determines the number of Test Cases, which should have beenexecuted if 99% of all functionality (system-wide) was Quality Assured; i.e. at aCalibration Factor of unity & Risk Quotient of 0%. The result of this calculation issubsequentlyutilised todetermine the actualCalibrationFactor (seeIndicator)&RiskQuotient(displayedastheCriticalValue)
xlvDescription1. Executedasabackgroundcalculation&representstheleanestTestingSolutionformost
situations(i.e.whenγ=δ)2. RelatestotheFunctionalProcessesDevelopedorModifiedfortheProjectexclusivelyxlviNote1. The scientific assessment regarding the probability of having found all Defects is
displayedasRiskMitigation2. When a Defect-Free Confidence value is enforced, the User overrides the scientific
assessmentinfavourofUserassessmentxlviiDisplayedasthe“CriticalValue”
www.testimation.com
Sunday,November5,2017 21
6. RiskMitigation1. Definition: The probability that all Defects were found in the Historical
SolutioniftheactualQualityAssuranceStrategyfollowedthescientificbasis(defaultAnalysis)definedabove
7. QualityOverrideSimilarity1. Definition: The Proportional similarity between Defect-Free Confidence &
RiskMitigationwithintheHistoricalSolution2. Description: The Proportional similarity between User perception &
scientificassessmentwithintheHistoricalSolution
5. RiskOverride
1. The Projected Solution utilises the Historical Solution as a frame of reference,scaling the Required Test Cases accordingly. After entering the appropriateinformationintheScopesection,theUsermustdecideifthedefaultAnalysis(i.e.at page load) is desirable. This section allows the User to override the defaultvalueofRiskQuotientintheProjectedSolutionbyusingtheslider&dial.xlviiiTheProjectedSolutionhastwopossiblepreconditions
1. TheHistoricalSolutionutilisesthedefaultAnalysisxlix2. TheHistoricalSolutionutilisesaUserSpecifiedValueofRiskQuotientl
2. CriticalValue1. Ifthe“RiskOverrideApplyCriticalValue”buttonisclicked,thevalueofRisk
Quotientpreciselycorrespondingto100(%)TestCoverageofDevelopmentis applied. The result relates to the Functional Processes Developed orModifiedfortheProjectedSolutionexclusively.ThisimplementstheleanestTestingSolution formostsituations{ε=ζ}.Hence,whenε>ζorε<ζ, the“Risk Override Apply Critical Value” button enforces the value of RiskQuotientsuchthatε=ζ
2. The Risk Override Critical Value may become negative (depending uponUser-Specifications);indicatingthattheHistoricalSolutionwasunder-testedfromtheperspectiveofscientificassessment
3. ExampleEstimate1. HistoricalSolution
1. Setup:UserStories=61,ExecutedTestCases=583,ConversionRatio=5,DIT’s=10,RiskQuotientSet-Point0%isappliedli
2. ThedefaultAnalysisindicatesthataninsufficientdegreeofTestingwasperformed(viatwomethods)lii
xlviiiThe scientific prediction of the Projected Solution based upon Statistical Mechanics,Probabilities & Engineering principles of Similitude, such that the default Analysisdetermines the number of Required Test Cases which should be executed if 99% of allfunctionality(system-wide)wasQualityAssured; i.e.ataCalibrationFactorofunity&RiskQuotientof0%xlixi.e.TheCriticalValueoftheRiskQuotienthasbeenappliedinQualityOverridelThe scaled result is subsequently utilised to determine the actual Calibration Factor (seeIndicator)lii.e.InsistingthattheHistoricalSolutionincorporated99%Defect-FreeConfidenceliiMethods1. TheTestExecutionApproach(viaCalibrationFactor)=CriticalPriorityTests;meaning
thatonlyaproportionofCriticalPriorityTestswereexecuted2. Graphicallybyγ<δ
www.testimation.com
Sunday,November5,2017 22
3. However, the User rejects the default Analysis & insists that the TestExecutionApproach(viaQualityOverride)was(inreality)Critical,High,Moderate&LowPriorityTests.Consequently,theUserwishestoapplythe Historical Solution as a baseline frame of reference to bereproducedintheProjectedSolution
2. ProjectedSolution1. Setup:UserStories=61,RiskQuotientCriticalValue=0%2. Because the User has overridden the default Analysis, the Projected
SolutionisconstrainedbytheHistoricalSolutionsuchthattheRequiredTest Casesmust equal the Executed Test Cases because the ProjectedSolution intends to mimic the perceived success of the HistoricalSolution
4. Defect-FreeConfidenceliii1. Definition:TheprobabilityoftheProjectedSolutionbeingDefect-Freebased
uponUserchoices2. ByselectingaRiskQuotientpre-setbuttonon theslider,orusing theRisk
Quotient dial to specify a value anything other than the Critical Valueindicatedbelowthedial,theUserclaimsthattheDefect-FreeConfidenceoftheHistorical Solutionwasdifferent to the scientific valuepredictedliv. Forexample, by selecting 0% Risk Quotient, the User seeks a Defect-FreeConfidenceof99%.Similarly, if theUser selects50%RiskQuotienton theslider,theUserseeksaDefect-FreeConfidenceof80.22%fortheProjectedSolution
5. RiskMitigation1. Definition: The probability of finding at least one Defect in the Projected
SolutionattheRiskOverrideconfigurationspecified6. RiskOverrideSimilarity1. Definition: The Proportional similarity between Defect-Free Confidence &
RiskMitigationwithintheProjectedSolution2. Description: The Proportional similarity between User perception &
scientificassessmentwithintheProjectedSolution
liiiNote3. The scientific assessment regarding the probability of having found all Defects is
displayedasRiskMitigation4. When a Defect-Free Confidence value is enforced, the User overrides the scientific
assessmentinfavourofUserassessmentlivDisplayedasthe“CriticalValue”
www.testimation.com
Sunday,November5,2017 23
6. GeneralConsiderations
1. The importance of knowing the Defect-Free Confidence & Risk Mitigationprobabilities (together) is for overall Risk Assessment & decision-making. Thesmaller the difference between these twoprobabilities, the lower theNetRisk&thegreaterthesimilaritybetweentheHistorical&ProjectedSolutions
2. Defect-FreeConfidenceDecisionTable
Defect-FreeConfidence TestCasesSuccessfullyPassed99%-97.43% Critical(all),High(all),Moderate(all)&LowPriority97.42%-93.15% Critical(all),High(all)&ModeratePriority93.14%-80.23% Critical(all)&HighPriority>0-80.22% CriticalPriority
3. TEstimator2UserScenarios1. TheUserapplieslvtheCriticalValueRiskQuotient to theHistoricalSolution&
the Risk Quotient of the Projected Solution is 0%. In this scenario, the Useracceptsscientificassessmentoverhumanrecollectionofhistoricalevents&theRequired Number of Test Cases for the Projected Solution is calculated at aDefect-FreeConfidenceof99%
2. TheUserappliesaNon-CriticalValueRiskQuotienttotheHistoricalSolutionlvi&theRiskQuotientoftheProjectedSolutionis0%.Inthisscenario,TheUseroverridesscientificassessmentoftheHistoricalSolution&declarestheDefect-FreeConfidencetobeaUserspecifiedvaluelvii
3. TheUserappliestheCriticalValueRiskQuotienttotheHistorical&ProjectedSolutions.Inthisscenario,TheUseracceptsscientificassessmentoverhumanrecollectionofhistoricalevents& intends toconfine theTestCoverage in theProjected Solution to the code under development only (testing the broadersystembeyondthecodeunderdevelopmentwillnotoccur).ThisimplementstheleanestTestingSolutionformostsituations{ε=ζ}
4. TheUserappliesaNon-CriticalRiskQuotientvaluetotheHistoricalSolution&theCriticalValueRiskQuotienttotheProjectedSolution
5. The User applies a Non-Critical Risk Quotient value to the Historical &ProjectedSolutions
4. TestCoverageofDevelopment1. Definition:TheproportionofDevelopmentcoveredbyTesting2. Description: The scientific assessment of the proportion of Development
coveredbyTesting.ThisdoesnotincludenorconsidertheperceivedcoveragebytheUser
lvThisisthedefaultsettingatpageloadlviTestCoverageofDevelopmentshouldbeconsideredpriortoinvokingthisactionlviii.e.TheDefect-FreeConfidenceindicatorontheleft
www.testimation.com
Sunday,November5,2017 24
6. HowtoexecuteaTEstimator3Estimate
A. Applicability
Uses UsersIn-houseProjects
1. BoardofDirectors2. ChiefExecutiveOfficers3. ChiefTechnologyOfficers4. BusinessOwners5. SecurityAgencies6. Corporations7. Government8. Military
The TEstimator3 Tool is a powerful & flexible instrument, facilitating the scaled
reproductionof anHistorical Solution to aProjectedSolution. Itmaybeused for any formofQuality Assurance, including Hardware as well as Software. The technique is based upon afundamental Engineering technique termed Similitude (typicallyappliedtoThermodynamics&Fluid Mechanics). A milestone for the development of Similitude was introduced by EdgarBuckingham via Dimensional Analysis (now known as Buckingham Π Theory). The basicprincipleofDimensionalAnalysis& thederivationof theΠGrouping techniqueDevelopedbyEdgar Buckingham, is such that a User (a Scientist or Engineer) may formulate an equationincorporating any number of unknown multidimensional variables, preceded by anexperimentally determined factor (traditionally expressed as “K”). The value of “K” may be aconstantorfunction,butitcanonlybedeterminedexperimentally&cannotbemathematicallyderived,onlyassignedanassumedvalue:
• Forexample1. “E=mc2”isaspecificsolutionof“E=K•mc2”where“K=1”2. Forthisexampleinreality,“K”isnotassumedtobeanyothervalue,onlyunity
A key feature of this technique is that there is no “wrong or right” combination of
multidimensional variables to combine; it is entirely the personal choice of the individualformulating the governing equation. This becomes particularly important in the TEstimator3Tool because its application is influenced by the logic being employed by the User. For thisreason,anotessectionhasbeenaddedsothatassumptions&otherrelevant informationmaybe captured& recorded at the time of estimation. The examples shown in “TEstimator3 UserScenarios”demonstratethataUsermayselectaparameterfromthedropdownlist&relateittoaperceived levelofHistoricalQualityAssuranceviathe“QualityOverride-RiskQuotient”.Bydoingthis,theUsercombinesallotherfactorsintoanundefinedexperimentalvalue(“K”).Theexperimentalvaluemaybedependentupon(forexample):
1. Publicholidays2. Resourceutilisation3. Sickleave4. Skillsets&/orSkillLevels5. Anyreasonableinfluenceorcombinationofinfluencesetc.
BecausetheUserisscalingfromanHistoricalSolutiontoaProjectedSolution,thevalueof
“K” is not required to be known. The effect of this is such that all the benefits & problemsnaturally incorporated into the Historical Solution’s experimental value “K” is also factoredinvisiblyintotheProjectedSolution.ThismeansthatwhatevernegativeinfluencestranspiredintheHistoricalSolution,arepresumedtooccurintheProjectedSolution.
www.testimation.com
Sunday,November5,2017 25
B. EstimationProcess
C. TEstimator3UserScenarios1. By example: Business Scenarios, Features, Integrated Systems, Interfaces, Number of
Screens,Reports,Requirements,Servers1. AUserwishestodetermineaRough-Order-of-MagnitudefortheQualityAssurance
EffortassociatedwithapotentialProject2. Aftersomeinvestigation,theUserdecidestouseanHistoricalSolutionasaframeof
Reference3. TheUserdecidestousethenumberofRequirementsastheirHistoricalEstimation
Basis4. The User selects Requirements from the dropdown list & inputs the number of
HistoricalRequirements&ExecutedTestCases5. The User inputs an estimate for the number of Requirements expected in the
ProjectedSolution;thisdoesnotneedtobeavaluewithhighconfidence(itmayberevisedatalatertime)
6. TheUserconsultswithrelevantstakeholders&assignsavalueof“QualityOverride-RiskQuotient”.Thevalueassignedisintendedtorepresentthesystem-wideDefect-FreeConfidenceoftheHistoricalSolution.ThisisnotjusttheDefect-FreeConfidenceof thespecificcode thatwasDevelopedorModified for theHistoricalSolution,butalsoitsimpactupontheexistingsystem&functionality.Apossibleindicatorforthisvaluemay be the standardRegression Suite; if the significantmajority of the TestCasesinthestandardRegressionSuitehavepassedinthenormalmanner,thentheDefect-Free Confidence of the Historical Solution should be high. If the standardRegression Suite experienced more failures than anticipated, then the “QualityOverride-RiskQuotient”valueshouldbesetlow.TheeffectofthisistoincreasetheTestingintheProjectedSolution.IftheUserbelievesthattheHistoricalSolutionwasashiningexampleofQualityAssurance,thenthe“QualityOverride-RiskQuotient”shouldbesetto0%
7. TheUserdecidestosetthe“RiskOverride-RiskQuotient”in-linewiththeavailabletime&budgetfortheProject;LowerRisk=moreTesting=greatercost
8. AProjectedValuefortheRequiredTestCasesiscomputed2. Byexample:Lines-of-Code,Modules,Objects
1. A User wishes to determine a Rough-Order-of-Magnitude for the number of UnitTests(TestCases)associatedwithapotentialProject
2. The User decides to the use Lines-of-Code of an Historical Solution as a frame ofReference
3. TheUser knows that “X” Lines-of-CodewereModified in theHistorical Solution&thattheProjectedSolutionwillcontainapproximately“Y”Lines-of-Code
4. TheUserknowsthat“X”Lines-of-Coderequired“Z”numberofUnitTests5. TheUserinterpretsthe“QualityOverride-RiskQuotient”(inthiscontext)toreflect
the system-wide coverage (influence) of the “X” Lines-of-Code upon the HistoricalSolution1. TheUserestimatesthatthe“X”Lines-of-Codetouchedapproximately20.33%of
system-widefunctionalityinsomeway(directlyorindirectly)
SetupScope
• HistoricalSolution• ProjectedSolution
SpecifyorCon�irmRiskQuotient
• HistoricalSolution• QualityOverride
• ProjectedSolution• RiskOverride
OutputEstimate
• ProjectedSolution
www.testimation.com
Sunday,November5,2017 26
2. The User selects a “Quality Override - Risk Quotient” of 90% using the RiskQuotientDial
6. TheUserdecidestosetthe“RiskOverride-RiskQuotient”in-linewiththeavailabletime&budgetfortheProject(lowerRisk=moreTesting=greatercost)
7. AProjectedValuefortheRequiredTestCases(UnitTests)iscomputed3. Byexample:SecurityThreats(pings)
1. AUserwishestodetermineaRough-Order-of-MagnitudeforthepotentialnumberofSecurityThreatsassociatedwithanewProject
2. TheUserselectsanappropriateHistoricalSolutiontoutilizeasaframeofreferenceforscaling
3. TheUserdescribes the structureofhis /her reasoning for selecting theHistoricalSolutionintheNotessection.Inthisexample,theUserdecidestorelatethenumberofaccessibleportstothenumberofpings
4. TheUserconcludesthat“X”numberofportsresultedin“Y”numberofpingsintheHistoricalSolution
5. TheUserisinformedthattheProjectedSolutionwillcontain“Z”numberofports6. TheUserinterpretsthe“QualityOverride-RiskQuotient”(inthiscontext)toreflect
thesystem-widepotentialforundetectedSecurityThreatsintheHistoricalSolution7. The User assumes a “Quality Override - Risk Quotient” of 21%; consequently, the
Useris95.81%confidentthatnoundetectedSecurityThreatsexistintheHistoricalSolution
8. The User decides to set the “Risk Override - Risk Quotient” in-line with a ZEROThreatpolicy&selects0%
9. Intheexampleabove1. FortheHistoricalSolution
1. TheSecurityThreatsfieldispopulatedby“X”(numberofports)2. TheExecutedTestCasesfieldispopulatedby“Y”(numberofpings)
2. FortheProjectedSolution1. TheSecurityThreatsfieldispopulatedby“Z”(numberofports)2. ThepotentialnumberofpingsiscalculatedintheRequiredTestCasesfield
(i.e.theexistenceofonepingisoneTestCase)
www.testimation.com
Sunday,November5,2017 27
7. Definitions
A. CalibrationFactor1. Dealswithsystem-widetests,notjustProjectscopetests2. This may include some or all of the Test Cases required to validate the
Functional Processes being Developed or Modified within the scope of theProject
3. Refer to the Quality Assurance Strategy Graph to determine if the selectedCalibrationFactorprovidestheappropriateTestCoverageofDevelopment
4. Set-Point0.25isselected1. AllCriticalPriorityTestsaretobeexecuted2. 80.22%probabilityoffindingatleastoneDefect
5. Set-Point0.5isselected1. AllCritical&HighPriorityTestsaretobeexecuted2. 93.15%probabilityoffindingatleastoneDefect
6. Set-Point0.75isselected1. AllCritical,High&ModeratePriorityTestsaretobeexecuted2. 97.43%probabilityoffindingatleastoneDefect
7. Set-Point1isselected1. AllCritical,High,Moderate&LowPriorityTestsaretobeexecuted2. 99%probabilityoffindingatleastoneDefect
8. Set-Point1.25isselected1. All Critical, High, Moderate & Low Priority Tests are to be executed +
25%Redundancy(25%moreteststhanSet-Point“1”requires)2. 99.6%probabilityoffindingatleastoneDefect
9. Set-Point1.5isselected1. All Critical, High, Moderate & Low Priority Tests are to be executed +
50%Redundancy(50%moreteststhanSet-Point“1”requires)2. 99.84%probabilityoffindingatleastoneDefect
10. Set-Point1.75isselected1. All Critical, High, Moderate & Low Priority Tests are to be executed +
75%Redundancy(75%moreteststhanSet-Point“1”requires)2. 99.93%probabilityoffindingatleastoneDefect
11. Set-Point2isselected1. All Critical, High, Moderate & Low Priority Tests are to be executed +
100%Redundancy(100%moreteststhanSet-Point“1”requires)2. 99.97%probabilityoffindingatleastoneDefect
B. Configuration12. SeeConversionRatio13. SeeDIT’sperTestCase
C. ConversionRatio14. AppliestoUseCases,UserStories&TestScenarios15. DenotestheaveragenumberofFunctionalProcessesassociatedwitheachofthe
above16. Examples
1. One (1) Use Case maps to five (5) Functional Processes (on average);thereforetheConversionRatioisfive(5)
2. One (1)User Storymaps to five (5) FunctionalProcesses (onaverage);thereforetheConversionRatioisfive(5)
www.testimation.com
Sunday,November5,2017 28
3. One(1)TestScenariomapstoone(1)FunctionalProcesses(onaverage);thereforetheConversionRatioisone(1)
D. CriticalValue17. ThevalueofCalibrationFactorpreciselycorrespondingto100TestCoverageof
Development(%)18. Relates to the Functional Processes Developed or Modified for the Project
exclusively19. Implements the leanest Testing Solution for most situations (unless otherwise
overriddendeliberatelybytheUser)20. CalculatestheprobabilityoffindingatleastoneDefectassociatedwiththeabove
E. Defect-FreeConfidence
21. HistoricalSolution1. Theprobabilityof theHistoricalSolutionbeingDefect-Freebasedupon
Userchoices22. ProjectedSolution
1. Theprobabilityof theProjectedSolutionbeingDefect-FreebaseduponUserchoices
F. DIT’sperTestCase
23. AnacronymforDynamicInformationTestsperTestCase24. The average number of fields per Test Cases containing dynamic information
whichmustbeverifiedwith“yesorno”confirmationbyahumanbeing25. Excludesallstaticcontent&fields
G. DynamicInformationTests(DIT’s)26. Themaximumnumberof“yesorno”Testsrequiredtobeexecutedbyahuman
being27. A “yes or no” Test may be described by the following question: is this field
correct,“yesorno”?28. Yes=TestPassed,No=TestFailed29. PartialorincompletePasscategorisationdoesnotapply30. These Tests are confined to Dynamic content only; Static content is not
applicable31. These Tests may include any combination of Positive & / or Negative Tests
required
H. ExecutedTestCases32. TheactualnumberofTestCasesExecuted
I. Framework
33. TestApproachfocusingonCalibrationFactor&RiskQuotient
J. FrameworkProbabilities
34. Avisual representationof the combinedeffectof theCalibrationFactor&RiskQuotient
35. Communicates the effect of the Calibration Factor & Risk Quotient upon theExpected (Targeting) Distribution of Functional Processes to be Developed orModified
www.testimation.com
Sunday,November5,2017 29
36. CommunicatesthattheUserexpectstofocusTestingaroundthemeannumberofDataFieldsperDatabaseRecord
37. CommunicatesthattheUserexpectstofocusTestingaroundthemostcommonlyusedsystem-wideFunctions
K. FunctionalProcess
38. A train of FunctionPoints enactedby theUser to execute aworkplace activity(i.e.theUserdoingtheirjob;WorkInstructions)
1. CreatingorDeletingCustomerAccountsetc.2. PrintingWeekly,MonthlyorAnnualSalesReportsetc.3. BillingCustomers,loggingCustomerComplaintsetc.
39. A suite of one ormore appropriate Functional Processes is termed a BusinessProcess
1. Example1:On-Boardinganewemployee(i.e.theBusinessProcess)mayinvolvethefollowingFunctionalProcesses
1. CreatinganE-MailAccount2. CreatingaUserApplicationAccount3. CreatingaFinancialAccount(salary,wageetc.)4. CreatingaSecurityPass5. CreatingUserAccess(aPass)totheBicycleCage
2. Example 2: Logging Customer Complaints is one Functional Process inthebroaderBusinessProcessofHandlingCustomerComplaints
L. ProgressionTesting
40. ReferstotestsrelatingtotheProjectCodebeingModifiedorDeveloped
M. QualityAssuranceArchitecture41. Theoverarching(system-wide)intendedapproachtoTesting
N. QualityAssuranceStrategy42. Theoverarching(system-wide)resultantapproachtoTesting43. ItutilisestheCalibrationFactor&RiskQuotientparametersdefinedbytheUser
intheQualityAssuranceArchitecturesection44. The key difference between Architecture & Strategy is that the Architecture
expressestheintended(desired)approachtoTestingviatheCalibrationFactor&Risk Quotient as independent parameters, whilst the Strategy expresses thecombinedeffectoftheCalibrationFactor&RiskQuotientupontheapproachtoTesting
45. TheQualityAssuranceStrategyGraphcommunicates1. Risk Exposure: the proportion of system-wide functionality being
DevelopedorModified2. RiskMitigation:theprobabilitythattheapproachtoTestingwill findat
leastoneDefect3. TestCoverageofDevelopment: theproportionofDevelopmentcovered
bytheTestApproach
O. QualityOverrideSimilarity
46. The Proportional similarity between Defect-Free Confidence & RiskMitigationwithintheHistoricalSolution
www.testimation.com
Sunday,November5,2017 30
P. RegressionTesting
47. ManualRegressionSuite1. Refers to a statistically random sampling (i.e. a subset) of the Manual
TestCasespredictedintheProgressionTestingsection2. Re-Execution of the statistical sample with a status of “Test Passed”,
yields 99% Confidence that the entire population of Manual Tests willpassiftheywereRe-Executed
3. Thisstatisticalapproach ispredominantlysuited tovery largeprojects;i.e.withthousandsofManualTestCases
4. Statistical approaches may be applied to provide cost effectivealternatives to the Quality Assurance of Software; particularly incompressedtimelines
48. AutomatedRegressionSuite1. Refers to a statistically random sampling (i.e. a subset) of the Manual
RegressionTestCasespredicted2. AutomatedRe-Executionof the statistical samplewith a statusof “Test
Passed”, yields 99% Confidence that the entire population of ManualRegressionTestswillpassiftheywereRe-Executed
3. Is appropriate if an intention exists to Re-Execute the subset multipletimes(typically:atleastfivetimes)
Q. RequiredTestCases49. ThenumberofTestCasesRequiredtobeExecuted
R. RiskExposure50. Istheproportionofsystem-widefunctionalitybeingDevelopedorModified51. Is governed by the number of Functional Processes under Development or
Modification(i.e.Scope)
S. RiskMitigation52. General:theprobabilitythattheQualityAssuranceStrategywillfindatleastone
Defect53. TheprobabilitythatallDefectswerefoundintheHistoricalSolutioniftheactual
QualityAssuranceStrategyfollowedthescientificbasis(defaultAnalysis)54. TheprobabilityoffindingatleastoneDefectintheProjectedSolutionattheRisk
Overrideconfigurationspecified
T. RiskOverrideSimilarity
55. The Proportional similarity between Defect-Free Confidence & RiskMitigationwithintheProjectedSolution
U. RiskQuotient56. Representstheacceptablelevelofrisk57. ReducesthecostoftestingastheRiskQuotientincreases58. ReducestheprobabilityoffindingDefectsastheRiskQuotientincreases59. CanbeappliedtomergetheTestApproach(CalibrationFactor)totheavailable
TestingbudgetinType1estimates.Thatis,thenumberofTestCaseswhichcanbeDesigned&Executedwithinthetime&financialconstraintsoftheProject
1. Decided by the User in isolation or consultation with the QualityAssuranceGovernancebody
2. Takestheform
www.testimation.com
Sunday,November5,2017 31
1. QualityBiasedTesting:Range=0-49%2. NeutrallyBiasedTesting:Singlecondition=50%3. RiskBiasedTesting:Range=51-100%
3. CriticalValue4. The value of Risk Quotient precisely corresponding to 100Test
CoverageofDevelopment(%)5. Relates to the Functional Processes Developed or Modified for the
Projectexclusively6. Implements the leanest Testing Solution formost situations (unless
otherwiseoverriddendeliberatelybytheUser)7. Calculates the probability of finding at least one Defect associated
withtheabove
V. Scope60. The number of Functional Processes to be Developed or Modified within the
contextofaproject
W. SystemUnaffected
61. Istheproportionofsystem-widefunctionalitynotbeingDevelopedorModified62. IstheresidualoftheRiskExposure(governedbytheScope)
X. UseCase
63. AsuiteofoneormoreFunctionalProcesses64. The term applied in WaterFall Methodology (historically), referring to the
mannerinwhichaUsershallexecuteaBusinessObjective.Thatis,howtheUsershallinteractwiththeFunctionsbeingDevelopedorModified
65. Typically, a well-written Use Case contains approximately five (5) TestScenarios;however,thisdependsuponmanyfactorsincludingthewritingstyle&experienceoftheUseCasecomposer.
66. ApoorlyconstructedUseCasewillcontaingreaterthanten(10)TestScenarios.Ifthisoccurs,theUsershouldconsiderdecomposingtheUseCaseintomultiplesmallerones
67. TheWaterFallanalogueofUserStorywithinAgileMethodology
Y. UserStory
68. AsuiteofoneormoreFunctionalProcesses69. ThetermappliedinAgileMethodology,referringtothemannerinwhichaUser
shallexecuteaBusinessObjective.That is,howtheUsershall interactwiththeFunctionsbeingDevelopedorModified
70. Typically, a well-written User Story contains approximately five (5) TestScenarios;however,thisdependsuponmanyfactorsincludingthewritingstyle&experienceoftheUserStorycomposer
71. A poorly constructed User Story will contain greater than ten (10) TestScenarios. If thisoccurs, theUser should considerdecomposing theUser Storyintomultiplesmallerones
72. TheAgileanalogueofUseCasewithinWaterFallMethodology
www.testimation.com
Sunday,November5,2017 32
Z. TestCase
73. ATestCaseiscollectionofDIT’stobeexecutedsimultaneously74. AnaverageTestCasecontainsanaveragenumberofDIT’sdefinedbytheUserin
theQualityAssuranceSection75. “Test Cases” refers to the maximum number of Test Cases that the Scope is
expectedtoyield76. “TestCase”referstomanualTestsonly;noautomatedTestingisapplicable
AA. TestCase&DefectComplexity
77. TestCaseComplexity1. Example: The average Complexity as configured by the User in the
QualityAssuranceArchitecturesection2. 10DIT’sperTestCaseconfiguredbyUser(DIT’s=DynamicInformation
Tests)3. 50DIT’sperTestCaseisthemaximumpermissiblevalueofthedial4. TestCaseComplexity=10/50=20%
78. DefectComplexity1. Refers to the complexity of its discovery, not the complexity of its
correction(fixing)bytheDevelopmentTeam2. It follows that the average complexity of a population of Test Cases is
directly proportional to the average complexity of the population ofDefectstheyraise
BB. TestScenario79. AsynonymforFunctionalProcess(typically)80. One(1)TestScenarioisequivalenttoone(1)FunctionalProcess(typically)81. A Use Case or User Story contains approximately five (5) Test Scenarios
(typically)82. Theconstitution&specificformofaTestScenarioissubjective&affectedbythe
experienceofthecomposer
www.testimation.com
Sunday,November5,2017 33
8. ApplicatorFunctionalProcesses
A. FunctionalProcessList
AFunctionalProcessisatrainofFunctionPointsleadingtoarequiredactionoroutputviaapathway;typicallythistakestheformofUserWorkInstructions.TheFunctionalProcessesforthis Application Suite are shown belowlviii. Each Functional Process begins with a jump-onpointlix& completes with a jump-off pointlx. The jumping-on points are QuEstimate View,LEstimate View, Basic View, Advanced View, TEstimator2, TEstimator3, REstimator (StandAlone) & @Risk (Stand Alone); the jumping-off point is always Save to File. Hence, the totalnumberofFunctionalProcesseswithintheApplicatorSuite(thisproductsuite) isshowntobetwentyfive(25),asfollows:
1. TEstimator11. QuEstimateViewàpathways
1. SavetoFile2. AEstimatoràSavetoFile3. AEstimatoràREstimatoràSavetoFile4. AEstimatoràREstimatoràPEstimatoràSavetoFile
2. LEstimateViewàpathways1. SavetoFile2. AEstimatoràSavetoFile3. AEstimatoràREstimatoràSavetoFile4. AEstimatoràREstimatoràPEstimatoràSavetoFile
3. BasicViewàpathways1. SavetoFile2. AEstimatoràSavetoFile3. AEstimatoràREstimatoràSavetoFile4. AEstimatoràREstimatoràPEstimatoràSavetoFile
4. AdvancedViewàpathways1. SavetoFile2. AEstimatoràSavetoFile3. AEstimatoràREstimatoràSavetoFile4. AEstimatoràREstimatoràPEstimatoràSavetoFile
2. TEstimator2àpathways1. SavetoFile2. AEstimatoràSavetoFile3. AEstimatoràREstimatoràSavetoFile4. AEstimatoràREstimatoràPEstimatoràSavetoFile
3. TEstimator3àpathways1. SavetoFile2. REstimatoràSavetoFile3. REstimatoràPEstimatoràSavetoFile
4. REstimator(StandAlone)àSavetoFile5. @RiskàSavetoFile
lviiiTotalnumberofFunctionalProcesses=4+4+4+4+4+3+2=25lixAstartingFunctionPointlxAfinishingFunctionPoint
4Pathways
4Pathways
4Pathways
4Pathways
4Pathways
3Pathways
2Pathways
www.testimation.com
Sunday,November5,2017 34
B. UseCase(UserStory)List
1. AUserexecutesaTEstimator1estimate2. AUserexecutesaTEstimator2estimate3. AUserexecutesaTEstimator3estimate4. AUserexecutesaStandAloneREstimatorestimate5. AUserexecutesaStandAlone@RiskVisualisationanalysis
• TotalnumberofUseCases(UserStories)=5
C. HowtoCalculate(orestimate)theConversionRatio
TheConversionRatiodenotestheaveragenumberofFunctionalProcessesassociatedwithUseCases,UserStories&TestScenarios.AlthoughUseCases,UserStories&TestScenarioshaveindustry definitions, no Universally accepted rigorously precise definition exists; Use Cases,User Stories & Test Scenarios are highly subjective & significant variation exists withincategories depending upon the composer. To overcome this practical impasse, theApplicatorProductSuitestandardisesontheuseofFunctionalProcessesinplaceofUseCases,UserStories&TestScenarios.ItachievesthisviatheConversionRatiosuchthattheUserassignsadefinitionofUseCases,UserStories&TestScenarios,intermsofthenumberofFunctionalProcesseachcategorypossesseslxi.ThedefaultvalueswithintheApplicatorSuiteare:
1. One(1)UseCasemapstofive(5)FunctionalProcesses(onaverage);thereforetheConversionRatioisfive(5)
2. One(1)UserStorymapstofive(5)FunctionalProcesses(onaverage);thereforetheConversionRatioisfive(5)
3. One(1)TestScenariomapstoone(1)FunctionalProcesses(onaverage);thereforetheConversionRatioisone(1)
4. Themathematicalrepresentationoftheapplicableformsareshownbelow
Inourcase:
𝐶𝑜𝑛𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑅𝑎𝑡𝑖𝑜 =𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑎𝑙 𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑒𝑠
𝑈𝑠𝑒 𝐶𝑎𝑠𝑒𝑠 𝑜𝑟 𝑈𝑠𝑒𝑟 𝑆𝑡𝑜𝑟𝑖𝑒𝑠=255= 5
Or
𝐶𝑜𝑛𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑅𝑎𝑡𝑖𝑜 =𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑎𝑙 𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑒𝑠
𝑇𝑒𝑠𝑡 𝑆𝑐𝑒𝑛𝑎𝑟𝑖𝑜𝑠=2525
= 1
lxiOnaverage,overapopulation
www.testimation.com
Sunday,November5,2017 35
D. UserWorkFlows(FunctionalProcessMap)