Experiences of a value assessment for products

SOFTWARE PROCESS IMPROVEMENT AND PRACTICESoftw. Process Improve. Pract. 2009; 14: 31–37

Published online 12 June 2008 in Wiley InterScience(www.interscience.wiley.com) DOI: 10.1002/spip.386

Experiences of a ValueAssessment for Products

Research Section

Pasi Ojala*,†

Hintanmutka 17 A 6, 90650 Oulu, Finland

In the near history, software process and product improvement has been recognized as ausable possibility to increase the quality of software development. Recently the costs and valueof software process improvement (SPI) as well as the cost-effectiveness and productivity ofsoftware development have become vitally important to many companies as competition hasbecome tougher.

The main purpose of this study is to collect experiences whether the value assessmentfor products support earlier-defined value-based approach and whether the value assessmentworks in practise, what are the strengths and weaknesses of using it and is it useful to companies.This is done by implementing value assessment in a case company step by step to study whichphases possibly work and which phases possibly do not work.

The industrial case shows that proposed value assessment for products is useful and supportsvalue-based approach. As well it demonstrates in practice, the strengths and weaknesses ofusing it and what should be taken into consideration when implementing it. Copyright 2008John Wiley & Sons, Ltd.

KEY WORDS: software engineering; software process and product improvement; value; worth; cost; value-based approach; valueengineering; value assessment

1. INTRODUCTION

The objective of the value-based approach (Ojala2006) is to explore ways to eliminate value lossin software development, software products, andsoftware process improvement (SPI) using the valueassessment framework of Koskela and Huovila(1997). In order to be able to determine thevalue in monetary terms, value-based approachuses economic-driven tools which are based oneconomic studies including, for example, the areasof cost estimation, cost calculation, and investmentcalculation.

∗ Correspondence to: Pasi Ojala, Hintanmutka 17 A 6, 90650Oulu, Finland†E-mail: [email protected]

Copyright 2008 John Wiley & Sons, Ltd.

Value-based approach has the most usable alleconomic-driven tools, which support determiningcosts over the entire life cycle of a productand process (for example life-cycle costing) andalso support both process and product-level cost-accounting like activity-based costing (ABC) isdoing when allocating costs to both processes andproducts. Value-based approach considers processand product-level cost accounting as necessitybecause products are the outputs of processesand, according to Kaplan and Cooper 1998, p. 3,for instance, there is a clear dependency betweenprocess and product costs. If product prices donot cover all production costs, the company sufferslosses.

The value-based approach prefers calculatingcosts instead of estimating them, and also considerssoftware development and SPI as investments, on

Research Section P. Ojala

which it is possible to spend too much moneyErdogmus et al. 2004 and Solingen 2004. In practiceit takes care that the customer requirements aremet in the best possible manner, ensuring quality,timeliness, and value in products as well as inprocesses, over their entire life cycle. In particularthe aim of ensuring quality connects it to the othermethods aiming for quality improvement.

The value-based approach indicates a cleardependency between the process and products. Itshows we need to develop and optimize processactivities so that processes produce the productsneeded. Furthermore, it sees that we must analyzeproducts in order to reveal problems in processesand develop processes from the product point ofview as well. This is vitally important, especiallyfor companies respecting customer opinions andaiming to optimize costs in their processes, becausethe customers are the ones paying for the prod-ucts and product-related services, and companieshave to allocate all costs to products to be able toprice them. The happier the customer is, the moreworth he sees in buying the products from us. Itis also clear that when we know our process andproduct costs, worth and value, our ability to esti-mate, budget, and control future risks will improvesignificantly.

The purpose of this study is to collect experiencesof using value assessment for products in anindustrial case. In more detail the purpose is toanswer the following questions:

• how the proposed value assessment for productsworks in practice,

• the strengths and weaknesses of value assess-ment for products,

• whether the value assessment for productssupport the value-based approach,

• whether the company assessed sees the valueassessment for products as useful.

2. VALUE ENGINEERING PROCESS

Even though there are several definitions in theliterature for the value engineering (VE) process,they all have similarities. Generally they state thatVE collects and analyzes value-related information,to create new ideas using the analyzed resultsand to evaluate and further develop them into ameaningful package, with the reduction of costs orthe increase of worth and improvement of value

as ultimate goals (Miles 1972, Kelly and Male 1993,Dell’ Isola 1997 and Park 1999).

This study categorizes VE process into threemain phases: prestudy (orientation), value study(information, function analysis, creativity, evalu-ation, development, presentation), and poststudy(monitoring, implementation). These phases areconsidered appropriate and justified since they con-stitute independent areas of VE presented by Miles1972, Kelly and Male 1993, Dell’ Isola 1997 and Park1999. As well these phases are in line with the VE’sgeneral purpose to collect and analyze value-relatedinformation, to create new ideas using the analyzedresults and to evaluate and further develop theminto a meaningful package, with the reduction ofcosts or the increase of worth and improvement ofvalue as ultimate goals (Ojala 2006). In his researchOjala 2006 has also outlined that these phases areusable especially for software engineering commu-nity.

According to VE, value is a measure – usually incurrency, effort or exchange, or on a comparativescale – which reflects the desire to obtain or retainan item, service or ideal. Cost is the price paid orto be paid. It can be divided into elements and, tosome extent, functions. Park (1999, 50) defines costas ‘an expenditure of money, time, labor, etc., toobtain a requirement.’ Worth is usually defined asthe lowest cost to perform the required function,or the cost of the lowest-cost functional equivalent.The most typical definition for value, is perhaps (1):

Value = WorthCost

(1)

where:

Value = The value of some object, product, serviceor process.

Worth = The least cost to perform the requiredfunction (product, service, or process), or the costof the least cost functional equivalent. If possible,it can also be the worth in money, what customersees in product, service or process.

Cost = The life-cycle cost of the object, product,service, or process (price paid or to be paid).

3. IMPLEMENTING A VALUEASSESSMENT FOR PRODUCTS

Value assessment for products was implementedin fall 2006. It was based on requirement lists and

Copyright 2008 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2009; 14: 31–37

32 DOI: 10.1002/spip

Research Section Value Assessment for Products

architectural component description lists. Togetherwith the requirement and component lists, severalother documents were analyzed during the assess-ment, including for example, different strategy andproject plans as well as different financial statementsand reports.

3.1. Information

The product to be assessed was a typical electronicproduct containing software and hardware. It wasdeveloped in collaboration by the vendor andthe customer. The implemented product-focusedassessment was supported and sponsored by thevendor’s and customer’s high-level management.In the assessment opening meeting, the purpose ofthe assessment was discussed with the vendor andthe customer. The definition value = worth/costwas discussed, and it was extremely important tofind out which requirements and components of theproduct gave the best value to the vendor withoutneglecting customer needs.

It was considered natural that too much detailin the architectural description would probablycause problems when calculating customer worth,because the customer does not necessarily haveenough technical expertise to understand the tech-nical product structure. Therefore in the assessmentan architectural list was provided which includedfunctional descriptions defining the activities foreach existing component. The vendor also empha-sized the importance of the component list becauseall resources were roadmapped using this list, andcost overruns could be more effectively analyzedusing the component list rather than the require-ment list. After the discussion it was decided thatvalue would be calculated for the requirementsdescribed in the product sales agreement and for thearchitectural components listed in the architecturaldescription.

The vendor emphasized that it would like toundertake the phases from creativity to presen-tation without the customer being present sincethese phases included brainstorming to gain a newunderstanding of all the processes used to developproducts. The customer saw that the most interest-ing phase for them was functional analysis, whereboth sides would prioritize requirements and com-ponents, and give estimates of worth and costusing relative numbers like percentages (not statingreal costs). The customer understood all wishes of

vendor and saw that they did not have a stronginterest in development methods and improvementproposals, which were considered to be more criticalfor the vendor’s business.

3.2. Function Analysis

In the first assessment meeting four customer rep-resentatives (referred to as ‘customers’) and threevendor representatives (referred to as ‘vendors’)prioritized the requirements and architectural com-ponents. Afterwards, the customers allocated worthto each requirement and component using a per-centage scale from 0 to 100%. The idea was toidentify in percentages what kind of worth the cus-tomer sees in the requirements and components.The vendors allocated costs using the same per-centage scale from 0 to 100%. As a result of this,the customers had given worth percentages for allrequirements and components, and the vendors hadgiven cost percentages for the same items. The cal-culated worth and cost were later compared, usingpercentages, to the real worth and cost, to find outthe difference between ‘belief’ and ‘reality’.

During the functional analysis phase the techni-cal representative of the customer pointed out thatwhen prioritizing, one cannot necessarily treat allcomponents equally, because some components aretied together. In practice certain components haveto be implemented before other ones. Some compo-nents are independent, and others are not. Certaincomponents rely on certain other components fortheir existence. However, he emphasized that eventhough this is the case, it does not affect all compo-nents, and prioritization clearly gives one a betterpicture of components and requirements, and oftheir importance in relation to each other.

All the interviewees agreed that the prioritizationof requirements and components clearly helped inthe next phase, in which the same requirementsand components were analyzed in terms of worthand cost. When asked to mark how much of thetotal price they would assign to each requirement,the customer representatives preferred to use per-centages rather than actual monetary values. Thevendors shared this viewpoint, and stated that itwas easier for them to give cost information inpercentages rather than in actual figures. As thefinal customer price and real production costs forrequirements and components were all known, it


DOI: 10.1002/spip 33


was decided that these allocations would also bedone, but for vendor use only.

The customers found it easy to assign worth totheir requirements, based on the customer price.The vendors also considered it easy to assigncosts to requirements. Both sides emphasized thatrequirements are easy to understand because theyare based on a clear, existing agreement. Only thearchitect found it slightly challenging to assigncosts to requirements, because resources are usuallyassigned to components and not to requirements.Therefore certain designers and their work cannotnecessarily be easily converted from the componentlevel to the requirement level. However he notedthat it is important to do this because this ishow cost and worth can be analyzed at the samelevel.

The results of requirement prioritizations wereunderstandable and expected among the cus-tomer and vendor representatives. Slight differ-ences existed, and these were discussed thoroughly.The customer found differences between how theirtechnical and user-oriented personnel saw require-ments. The vendor also found differences betweenthe project management’s and the technical person-nel’s comments. It seemed that the amount of tech-nical knowledge gave more logical reasoning forunderstanding the implementation of requirementsas components. By comparing the customer’s andvendor’s averages it was also possible to identifysome significant differences between their respec-tive priorities. For example, requirements such asservices were clearly more important for the vendorthan for the customer. The discussion of how thetwo parties could be part of the same project andyet understand each others’ interests so surprisinglywrongly was extremely profitable.

One conclusion of discussions was that worthand cost allocations for all requirements andcomponents were relevant for both sides, even ifonly stated as percentages. According to customerthey also had their own idea about the actualcosts of production, and since they knew the worththey were satisfied. Figure 1 presents the averageworth and cost for requirements. In this figure wecan observe how, for example, the customer hasevaluated the worth of picture call function as beingnoticeably higher than the vendor’s estimation of itsproduction cost. In practice this means value for thevendor. Figure 2 presents the average worth andcost for components.

Worth & Cost in Requirements

0.0

5.0

10.0

15.0

20.0

25.0

30.0

Pictur

e ca

ll

Emer

genc

yUse

r

Serve

r

Distan

ce co

nfigu

ratio

nVide

o

Servic

e

Camer

a

Activit

ies

Requirement

%

C AV Worth (%) V AV Cost (%)

Figure 1. Average worth and cost for requirementsincluding all interviewees (AV, average, C, customer,V, vendor)

Worth & Cost in Architecture

0.0

5.0

10.0

15.0

20.0

25.0

30.0

Basic

struc

ture

Settin

gs Log

Teleco

mm

Video

Emer

genc

y

Serve

r

Users

Distan

ce co

nfig

Sendin

g

Activit

ies

Surve

illanc

e

Servic

es

Component

Prio

rity

C AV Worth (%) V AV Cost (%)

Figure 2. Average worth and cost for componentsincluding all interviewees (AV, average, V, vendor)

On the whole the experiences of using prioritiza-tion in ranking requirements and components werepositive. Even more interest was seen in the analysisof worth and cost for each requirement and com-ponent, and especially in the differences identifiedbetween customer and vendor, as well as betweentechnical- and user-oriented personnel.


34 DOI: 10.1002/spip


Table 1. Evaluated improvement proposals

Systemstability

Safety Optimizedfunctioning

Easy touse

Maintainability Profitability Total

Estimation 25.0 40.0 15.0 80.0 15.0 75.0 250.0Multiprocessing 125.0 20.0 22.5 20.0 30.0 12.5 230.0Technical environment 75.0 80.0 37.5 100.0 60.0 25.0 377.5Architectural plan 150.0 100.0 30.0 40.0 75.0 50.0 445.0Design plan 100.0 120.0 45.0 120.0 90.0 37.5 512.5Project management process 50.0 60.0 7.5 60.0 45.0 62.5 285.0

525.0 420.0 157.5 420.0 315.0 262.5 2100.0

3.3. Creativity

In accordance with the agreement between the cus-tomer and the vendor, only the vendor participatedin the phases from creativity to presentation. Thefirst step in the creativity phase was to allocate coststo all requirements, and then to all components.According to the vendor it was easy to allocate coststo the requirements and components. General costswere perhaps the most difficult costs to allocate.This was because costs such as the project man-ager’s salary usually cannot be allocated directly toany particular requirement or component.

After cost allocations had been completed, theproject team started brainstorming. The vendorsevaluated priority lists, figures, and worth and costcalculations for all requirements and components.All personnel were encouraged to explain how theywould improve value at both requirement andcomponent levels. According to their comments,clear figures helped a lot in understanding wherethe most significant differences in value existed. Onthe basis of the figures it was noted that certainrequirements and components did not create goodvalue. After discussion the project members sharedthe opinion that this was because of the unfinishedarchitectural plan. This had an influence on theplanning and design of these items and thus theyhad been delayed, and created significantly highercosts.

Project members could also observe from thecharts presented how time-consuming it was tostart using new technical environments withoutgood planning. The new technical environmentdelayed the implementation of certain require-ments and components significantly. New technicalchallenges, such as developing software for multi-processor environments, were also named as onereason for delays. This was because project person-nel did not have sufficient training in working in the

multiprocessor environment. As a result of all theproblems mentioned, working hours were about15% higher than expected, and three componentswere not implemented at all.

3.4. Evaluation

At the beginning of the evaluation phase theproject team discussed criteria for the evaluationof improvement ideas. The criteria decided on weresystem stability, safety, optimized functioning, easeof use, maintainability, and profitability. First allthe project team members were asked to givea relative percentage (maximum 100%) for howimportant each criterion was for their project.Second, the project personnel calculated averagesfor all the criteria. The calculated averages were asfollows: system stability 25%, safety 20%, optimizedfunctioning 7.5%, ease of use 20%, maintainability15%, and profitability 12.5%. After thus defining theweightings of the criteria, the project personnel gavepoints to each improvement proposal on a scale ofone to six, where six indicated maximum pointsand one, minimum. The points allocated weremultiplied by the calculated weighting percentages.The results are in Table 1 as follows.

The project team discussed these results. The mostsurprising result was that the importance of thetechnical environment was as high as third place.Problems in design and architectural planningwere expected, as were problems related to projectmanagement. Estimation and multiprocessing gotthe least points, so their importance to the projectwas not considered to be as high. However, it wasnoted that if the project would have been morebusiness critical this would not have been the case.The more business critical the project would havebeen the more weighting the profitability criterionwould have got.


DOI: 10.1002/spip 35


3.5. Development

In the development phase, the biggest improvementideas were separately developed further, in orderto examine their practical implications. Each ideadeveloped included issues such as description,positive consequences, negative consequences andpotential cost savings.

The project personnel stated: ‘It has been difficultto get the necessary working resources for smallprojects.’ The architecture and design phases haveperhaps suffered from this the most. There had notbeen enough time to review these phases, whichcan be noted in the presence of incomplete plans.Both plans had been updated several times duringthe writing of code, which had sometimes stoppedcoding for several days. The project team calculatedthat if there had been support resources for makingmore comprehensive plans and reviewing them, theproject would have been 200 working hours shorter.The potential cost savings would have been about18,000.

At the moment the ability to use the existingcharacteristics of technical tools is weak. Theuse of pre-existing components is also ratherpoor. The result is that code has to be writtenfrom start to finish each time. The project groupevaluated that if basic components for developmentwork had existed, 80 fewer working hours wouldhave been required. If there had been sufficienttechnical training concerning the new environments(dotNET and ATL 7) for key personnel, 140 fewerworking hours would have been required. Intotal, the potential cost savings would have beenapproximately 10,000.

From a project management point of view, itis problematic that all the employees are alwaysassigned one hundred percent to a given project.As a consequence, there is not enough supportavailable if needed, and ‘the wheel is inventedseveral times in different projects.’ The projectteam evaluated that with satisfactory support inevaluating the architectural plan, the design plans,and the extra need for time in starting to use newtechnologies, 90 fewer working hours would havebeen required. In financial terms, this would havemeant a saving of about 8000.

3.6. Presentation

The results of the product-focused value assessmentwere presented phase by phase to the high-level

management. The project team supported thepresentation by giving brief comments. In thepresentation a clear emphasis was placed onpresenting customer needs and wants, and thecorresponding costs to the company. The valueindexes were used to outline the existing value-increasing opportunities. The potential cost savingproposed was approximately 20% of product price.

After the presentation had ended the manage-ment wanted to discuss the value improvementopportunities presented with the project person-nel. Some improvement ideas were implementedand some were developed further; others werepostponed due to lack of resources. As a whole,the assessment strongly emphasized collaborationbetween the customer and the vendor, and all theimprovement proposals were in line with the cus-tomer’s interests as well.

4. CONCLUSIONS

All participants agreed that the assessment workedwell and assessment process was clear and practi-cal. The product assessment was considered to besignificantly more effective than the process assess-ment. It was as well considered to represent thebusiness point of view as the customer participatedto the assessment.

The product-focused assessment had severalstrengths. It gave more customer-oriented improve-ment proposals than process assessments andproduct-related improvement was the languagethat the customer understood and was in a way‘buying’. For the customer it was important to par-ticipate in decisions about which features wouldbe implemented and which would not. The cus-tomer also wanted to prioritize product componentsand considered it important that the vendor-askedquestions what should and should not be done.Company considered it important that when theassessment is undertaken together with the cus-tomer, it can keep the customer more satisfied,which is a good basis for business. It was alsoemphasized that if value assessment is done in theplanning phase of a product, it is cheaper for anycompany than making changes after several monthsof development work.

Value assessment for products supports value-based approach to software engineering in severalways. The comments from Company show that


36 DOI: 10.1002/spip


there exists a practical need for enhancement of thescope of software engineering, in a more value-driven direction. This seems to be reasonable,especially if the company in question wants tocalculate costs in order to plan product-relatedactions before implementing them. In practice, bothcompany and their customer showed a specialinterest in this kind of planning, from which bothparties get value.

However there are several limitations to thisarticle. First, not all VE practices have been usedin the software engineering area as they have beenused in other industrial areas. As well the empiricalfindings of this study are rather limited as thisstudy bases on one industrial case and due to thelimitations in the cost-accounting system some costswere estimated in this study as well. Furthermore,value assessment, as for example many capability-maturity-based assessment methods seem to basealso on a heuristics and industry best practicesrather than a solid theory.

REFERENCES

Dell’ Isola A. 1997. Value Engineering: PracticalApplications. . . for Design, Construction, Maintenance &Operations. R.S Means Company, Inc: Kingston, MA.

Erdogmus H, Cusumano MA, Kontio JG, Raffo D. 2004.The sixth International Workshop on Economics-DrivenSoftware Engineering Research (EDSER-6). Proceedings ofthe 26th International Conference on Software Engineering(ICSE 04). IEEE Computer Society: Edinburg, 761–762.

Kaplan R, Cooper R. 1998. Cost & Effect. Using IntegratedCost Systems to Drive Profitability and Performance. HarwardBusiness School Press: Harward.

Kelly J, Male S. 1993. Value Management in Design andConstruction. E & FN Spon: New York.

Koskela L, Huovila P. 1997. On Foundations of ConcurrentEngineering in Construction CEC’97. London, 3–4 July. TheInstitution of Structural Engineers: London, 22–23.

Miles L. 1972. Techniques of Value Analysis and Engineering.McGraw-Hill: New York.

Ojala P. 2006. Implementing a value-based approachto software assessment and improvement. Doctoraldissertation, University of Oulu.

Park R. 1999. Value engineering. A Plan for Invention. St.Lucie Press: New York.


DOI: 10.1002/spip 37

Documents

Experiences of a value assessment for products